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FOREWORD 

In  January  1961,  General  Curtis  E.  LeMay  expressed  concern  over  the  steadily  increasing  costs  of 
new  weapon  systems.  He  directed  the  Inspector  General  to  conduct  a  series  of  surveys  of  major  Air 
Force  contractors’  management  policies,  procedures,  and  practices,  and  the  costs  generating  therefrom 
which  result  in  charges  against  Air  Force  contracts.  He  also  desired  that  management  efficiencies  and 
economics  be  cited  and,  wherever  possible,  be  given  wide  aerospace  industry  dissemination.  In  com¬ 
bination  with  that  activity,  we  in  the  Air  Force  Systems  Command  (AFSC)  began  critical  reappraisal 
of  our  own  activities.  The  AFSC  appraisal  culminated  in  a  number  of  System  Program  Management 
Surveys  (SPMS),  which  reviewed  both  Air  Force  and  contractor  management  of  specific  system  programs 

During  1962,  the  USAF  Inspector  General  and  AFSC  completed  surveys  of  a  representative  group 
of  Air  Force  contractors.  This  pamphlet  provides  a  summary  of  the  more  significant  lessons  learned 
from  surveys  of  twenty-four  major  contractors.  For  those  who  attended  the  Monterey  Management 
Conference,  many  of  the  topics  in  this  pamphlet  will  be  familiar.  The  corrective  actions  reported  are 
not  offered  as  the  final  solutions  or  the  only  solutions  for  these  problems.  Nor,  for  that  matter,  are  the 
causes  which  are  identified  with  the  problems  considered  to  be  absolute  and  unqualified  in  every  sense. 
In  some  cases  the  corrective  measures  have  proved  satisfactory,  others  have  yet  to  complete  or  undergo 
trial  periods  and  evaluation.  They  are  typical  of  some  of  the  actions  taken  by  industry  and  the  Air 
Force — not  the  whole  story  of  what  is  being  done  in  these  areas.  The  many  excellent  accomplishments 
and  management  practices  reported  by  survey  teams  warranted  expeditious  dissemination  apart  from 
this  pamphlet. 

A  candid  treatment  such  as  this  may  permit  the  drawing  of  incorrect  inferences  as  to  the  general 
efficacy  of  industry/Air  Force  management,  particularly  by  those  uninitiated  to  the  complexity  of  some 
of  the  common  deficiencies.  The  complexity  of  the  interrelations  which  must  be  considered  in  resolving 
these  problems  is  the  principal  reason  that  more  satisfactory  permanent  solutions  have  not  been  found 
long  ago.  Improvements  require  emphasis  by  management  at  all  levels.  The  extent  to  which  these  same 
problems  repeatedly  occur  should  eliminate  any  complacency  or  rationale  on  the  part  of  any  manager 
that  such  shortcomings  apply  only  to  others. 

To  the  managers  in  the  aerospace  industry,  it  is  my  earnest  desire  that  you  assess  your  own  activi¬ 
ties  in  the  light  of  the  lessons  contained  in  this  pamphlet. 

Q 

B.  A.  SCHRIEVER 
General,  USAF 
Commander 
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Systems  Management 

A  SUMMARY  OF  LESSONS  LEARNED  FROM  AIR  FORCE  MANAGEMENT  SURVEYS 

This  pamphlet  is  presented  as  an  aid  for  industry  and  Air  Force  Managers  concerned  with  System  Manage- 
|  ment  or  managment  functions  related  to  Air  Force  contracts.  It  summarizes  the  AFSC  analysis  of  re- 
\  petitive  management  deficiencies  found  by  Air  Force  surveys  of  major  contractors,  and  provides 
p  information  concerning  the  corrective  actions  by  contractors  and  the  Air  Force. 

NOTE:  Send  recommended  changes  to  this  pamphlet  to:  Hq  AFSC  (SCS-8),  Andrews  AFB,  Washing¬ 
ton  25,  D.  C. 
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CHAPTER  I 

Introduction 


WHAT’S  THE  SUBJECT? 

A  Recurring  management  deficiencies  that  are: 

•  Common  to  companies  working  under  Air  Force  contracts 

•  Problems  of  joint  concern  to  the  Defense  Industry  and  the  Air  Force 

WHY  THIS  PAMPHLET? 

A  To  inform  Air  Force  and  Industry  management  of  the  nature  of  the  deficiencies  repeatedly  found  by 
management  survey  teams 

A  To  indicate  the  causes  as  interpreted  by  the  AFSC  analyses 
A  To  report  the  corrective  measures  taken  by  some  contractors  and  the  Air  Force 

A  To  stimulate  Air  Force  and  Industry  managers  to  take  a  critical  look  at  their  own  activities  to  pre¬ 
vent  or  correct  similar  problems 

WHO  PREPARED  IT? 

A  The  Office  of  the  Assistant  for  Management  Surveys,  DCS/Systems,  assisted  by  other  staff  offices  of 
the  Headquarters,  Air  Force  Systems  Command. 

HOW  WERE  DEFICIENCIES  REPORTED? 

A  By  Air  Force  management  surveys  of  two  types: 

•  Industrial  Management  Assistance  Surveys  (IMAS)  conducted  by  the  USAF  Inspector  General  (IG) 
by  direction  of  the  Secretary  of  the  Air  Force,  which  are — 

—  Surveys  of  major  AF  contractors  to  evaluate  their  management  effectiveness  in  fulfilling  con¬ 
tractual  obligations. 

—  Detailed  examinations  of  a  contractor’s  total  AF  contractual  activities,  usually  at  one  location. 

•  System  Program  Management  Surveys  (SPMS)  performed  by  AFSC  personnel  by  direction  of  the 
Commander  AFSC  are — 

—  Surveys  of  specific  system  acquisition  programs,  or  portions  thereof,  to  evaluate  management 
effectiveness,  and  may  include  cognizant  AF  organizational  elements,  prime  or  associate  con¬ 
tractors  and  subcontractors. 


WHAT  WAS  THE  SCOPE  OF  CONTRACT  ACTIVITIES? 

A  The  twenty-four  contractors  surveyed  are  a  cross-section  of  major  electronic,  aeronautical,  ballistic, 
and  space  systems  producers. 

*  Contractual  obligations  ranged  from: 

—  Companies  responsible  for  integration  and  turnover  of  complete  operational  system,  to — 
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—  Companies  responsible  only  for  equipment  design,  development  and  production. 

•  Types  of  contracts  included: 

—  Cost  plus  fixed  fee 

—  Cost  plus  incentive  fee 

—  Fixed  price 

•  AF  contracts  had  high  value,  generally  above  $25  million. 

ARE  ALL  IMPORTANT  MANAGEMENT  PROBLEMS  INCLUDED?  NO. 

There  exists,  for  any  given  time  period,  a  set  of  causual  factors  tending  to  destabilize  the  industry/ 
government  relationship.  This  condition  is  generated  by  the  conflicting  viewpoints  about  management 
concepts  which  always  prevail  to  some  degree  within  government  circles.  The  dominant  viewpoints 
upon  which  management  policy  is  baaed  in  the  various  governmental  departments  (DOD,  NASA,  etc) 
may  at  times  be  in  conflict.  Similarly,  conflicting  viewpoints  may  dominate  intra -depart mental  policy 
(DOD  or  within  a  Service  Department).  Such  conflicting,  dominant  viewpoint*  have  resulted  in  incon¬ 
sistencies,  not  only  about  the  respective  management  roles  of  the  contractor  and  the  government  con¬ 
tracting  agency,  but  also  on  other  facets  of  program  and  functional  management.  The  broader  aspects 
of  this  problem  are  obviously  beyond  the  purview  of  the  Air  Force  to  deal  with  unilaterally;  therefore, 
such  factors  were  not  explored  in  the  problem  analyses  of  this  pamphlet.  However,  the  effects  of  in¬ 
consistent  application  of  Air  Force  policies,  are  included  in  the  summary  analysis. 

It  suffices  here  to  recognize  the  broad  problem  and  to  acknowledge  that  some  defense  industry  leaders 
feel  there  can  be  no  real  solution  to  the  cost  estimation — cost  overrun  problem  until  the  government 
achieves  greater  consistency  in  its  concepts  of  management. 
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CHAPTER  II 

Analyses  of  Management  Repetitive  Deficiencies 

SECTIONS 

1.  Program  and  Contract  Functions 

II.  Engineering  Functions 

III.  Production  and  Quality  Assurance  Functions 

IV.  Purchasing  and  Materiel  Functions 

V.  Product  Support  Functions 


3 


1  June  1963 


AFSCP  375-2 


NOTICE  TO  THE  READERS 


This  report  is  intended  primarily  for  top-level  Air  Force/Industry  managers  who 
are  busy  people,  not  likely  to  spend  much  time  on  another  lengthy,  difficult-to- 
read  report.  Generally,  they  are  quite  familiar  with  the  topics  to  be  covered 
here.  The  format  favors  brevity  over  detailed  treatment,  accepting  that  some 
intelligence  is  lost,  misinterpretations  may  occur,  and  the  summary  format  may 
not  be  useful  to  people  unfamiliar  with  the  problems. 

Each  deficiency  summarized  is  common  to  several  contractors  whose  organizations 
and  contract  obligations  may  differ  greatly.  The  degree  of  seriousness  varies 
with  the  conditions  relevant  to  each  contractor.  The  symptoms  listed  are  mani¬ 
festations  of  a  given  problem,  usually  side  effects  or  after  effects,  but  some  may 
in  turn  be  causes  of  other  problems.  They  have  been  determined  from  analysis 
of  each  contractor’s  specific  case,  and  are  not  necessarily  applicable  where  condi¬ 
tions  differ  from  those  analyzed. 

The  causes  reported  for  a  given  problem  are  determined  by  analysis  of  specific 
conditions  relevant  to  each  contractor  having  the  problem.  They  are  not 
necessarily  applicable  in  entirety  to  any  individual  contractor.  Similarly,  solu¬ 
tions  by  contractors  for  a  given  problem  are  based  upon  those  contractors’ 
judgments  as  to  preferred  corrective  actions  under  the  relevant  conditions.  They 
are  not  the  only  solutions  that  can  be  applied  under  the  identical  conditions, 
and  not  necessarily  appropriate  for  conditions  different  from  those  analyzed. 
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SECTION  I 

Program  and  Contract  Function 

Finding  No.  Pngt  No. 

1.  Decentralized  Program  Management  Lacked  Essential  Controls .  9 

2.  Deficiencies  in  Configuration  Management  .  11 

3.  Late  Definitization  of  Letter  Contracts .  17 

4.  Cost  Control  and  Accounting  Deficiencies .  19 

5.  Make-or-Buy  Policies  Not  Enforced  .  21 

6.  Costs  are  Underestimated .  23 

7.  Ineffective  Management  Audit .  25 

8.  Limited  Standards  for  Manpower  Utilization .  27 

9.  Deficient  Reports  Control  Procedures .  29 


7 


1  June  1963 


AFSCP  375-2 


SUBJECT 


FINDING  No.  1 


DECENTRALIZED 

PROGRAM 

MANAGEMENT  LACKED 
ESSENTIAL  CONTROLS 


IN  CONTRACTOR  ORGANIZATIONS  THAT  WERE  STRUC¬ 
TURED  ACCORDING  TO  FUNCTIONAL  LINE  DEPARTMENT 
CONVENTIONS,  TOP  MANAGEMENT  DID  NOT  TAKE  ACTION 
TO  INSURE  THAT  INTERNAL  POLICIES,  PROCEDURES,  AU¬ 
THORITY  AND  RESPONSIBILITIES  WERE  CLEARLY  DE- 
FINED  FOR  INTEGRATED  PROGRAM  CONTROL. 


Symptoms  of  Basic  Deficiencies: 

•  Uncertainty  or  confusion  on  the  part  of  line  organization  managers 
as  to  exact  responsibility  and  authority  on  program  matters. 

•  Program  decisions  are  not  made  in  timely  manner  or  not  imple¬ 
mented  promptly. 

•  Program  financial  control  is  fragmented  among  various  line  organi¬ 
zations,  program  financial  accounting  weak  or  non-existent. 

•  Engineering  decisions  are  made  unilaterally  by  design  groups  with 
little  regard  for  total  program  effects. 

Fundamental  Causes: 

•  Corporate/division  official,  or  level  at  which  semi-autonomous  line 
organizations  report,  does  not  have  overview  of  essential  elements 
of  the  program;  SPD  must  go  to  several  departments  to  obtain 
answers  to  program  matters. 

•  Nominal  program  manager  does  not  have  true  control  over  pro¬ 
gram  finances,  schedule,  design  decisions,  or  design  change  decisions. 

•  Contractor’s  inter-  and  intra-organization  policies  and  procedures 
do  not  insure  integrated  program  management  by  the  semi¬ 
independent  line  organizations. 

-  Vested  interests  prevail  rather  than  decisions  in  program  best 
interest. 

-  Conflicts  between  functional  managers  not  readily  brought  up  to 
higher  level  for  timely  resolution. 

-  Duplication  between  functional  organizations. 

-  Difficulty  in  determining  when  breakdown  in  program  coordina¬ 
tion  or  communication  has  occurred  between  parallel  organiza¬ 
tions  until  a  serious  problem  occurs. 

-  Lack  of  program  familiarity  and  level  of  detailed  program  knowl¬ 
edge  at  corporate  decision-making  level  (above  semi-independent 
departments)  increases  likelihood  of  erroneous  program  decisions. 

-  Strong  policy  and  procedural  discipline,  essential  to  achieve 
effective  program  management  by  semi-autonomous  organizations, 
is  not  enforced  or  even  defined  in  some  cases. 

-  Lack  of  integrated  program  planning. 


9 


AFSCP  375  2 


1  June  1963 


Actions  By  Contractor: 

•  Assignment  of  a  program  manager  and  vertical  program  organiza¬ 
tion  to  assure  financial,  schedule  and  design  control.  Program  au¬ 
thority  clearly  vested  in  the  program  manager. 

•  On  programs  where  vertical  program  organization  was  uneconomical 
or  otherwise  not  feasible,  inter-  and  intra -departmental  policies  and 
procedures  revised  to  improve  program  control  and  decision-making. 
In  all  cases,  a  central  configuration-control  board  was  established 
with  all  program-active  departments  represented. 

•  Organizational  discipline  enforced  rigorously. 

Actions  By  Air  Force: 

•  Established  clear  cut  management  interfaces  between  SPO  and  com¬ 
pany  management;  reporting  procedures  from  contractor  to  SPO 
better  defined. 

•  In  a  few  cases  the  contractor  was  requested  to  establish  a  program 
manager  and  vertical  organization  for  program  management  on  high 
priority  programs  of  large  magnitude,  or  where  contractor  had  a 
mixture  of  efforts  on  AF  and  non-AF  programs  with  CPFF  and  fixed 
price  contracts. 

•  Require  program  management  plan  and  resumes  of  key  program 
management  individuals  from  bidders  responding  to  Air  Force  Re¬ 
quest  for  Proposal  (RFP)  as  a  part  of  the  proposal  package  on  new 
programs. 
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SUBJECT 

DEFICIENCIES  IN 
CONFIGURATION 
MANAGEMENT 


FINDING  No.  2 

INADEQUATE  MANAGEMENT  OF  THE  CONFIGURATION  OF 
EACH  END  ITEM  OF  HARDWARE  TO  BE  DELIVERED  TO 
THE  AIR  FORCE  MAKES  LOGISTIC  SUPPORT  DIFFICULT, 
COSTLY,  AND  COMPROMISES  THE  OPERATIONAL  EFFEC¬ 
TIVENESS  OF  THE  SYSTEM.  DIFFICULTIES  IN  THE  EARLY 
BALLISTIC  MISSILE  PROGRAMS  LED  TO  RECOGNITION  BY 
THE  AIR  FORCE  AND  BALLISTIC  SYSTEM  CONTRACTORS 
THAT  NEW  POLICIES,  IMPROVED  TECHNIQUES  AND  PRO¬ 
CEDURES  WERE  VITALLY  NEEDED  FOR  CONFIGURATION 
MANAGEMENT.  THE  NEW  METHODS  THUS  DEVELOPED 
WERE  FIRST  APPLIED  TO  THE  BALLISTIC  SYSTEM  PRO¬ 
GRAMS. 

THE  PAST  YEAR  HAS  BEEN  ONE  OF  PROGRESSIVE  EXTEN¬ 
SION  OF  THESE  METHODS  TO  ALL  AIR  FORCE  SYSTEMS / 
EQUIPMENTS. 


Symptoms  of  Basic  Deficiencies: 

•  Inabilitv  to  determine  exact  end  item  hardware  configuration  from 
technical  documentation  at  time  of  shipment  from  factory. 

•  Inability  to  determine  status  of  changes  incorporated  at  field  loca¬ 
tions  or  status  of  changes  being  processed,  or  status  relative  to  ap¬ 
proved  changes. 

•  Failure  to  meet  program  milestones  for  installation,  check-out  and 
system  integration. 

-  Inability  to  forecast  work-to-completion  of  work  tasks — schedule 
unrealistic. 

•  Technical  Orders/Manuals  not  updated  to  latest  hardware  changes. 

•  Spares  not  updated — incompatible  with  operational  or  R&D  system. 

•  Excessive  time  and  effort  required  to  isolate  design  deficiencies  from 
failed  or  malfunctioning  components  resulting  from  partial  incorp¬ 
oration  of  total  change,  out-of-phase  incorporation  of  sequenced 
change;  or  improper  operation  or  maintenance  resulting  from  pro¬ 
cedures  not  updated  to  latest  hardware  change. 

•  Technical  description  documentation  late,  inadequate  for  Air  Force 
acceptance  of  integrated  system. 

•  Hardware  configuration  identification  and  accounting  status  at  turn¬ 
over  inadequate  to  operate,  maintain  and  logistically  support  opera¬ 
tional  systems. 

•  Increased  program  costs. 

-  Contract  cost  overruns. 

-  Program  schedule  slippages. 
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Fundamental  Causes: 

•  Lack  of  full  understanding  by  SPO  of  total  technical  description 
required  for  the  Air  Force  to  operate,  maintain  and  logistically  sup¬ 
port  operational  system.  As  a  result: 

-  Contractual  requirements  for  configuration  management  by  con¬ 
tractor  not  definitive. 

-  Documentation  required  for  configuration  identification,  control 
and  status  accounting  not  defined. 

-  Omitted  essential  elements  of  management  and  safeguards  em¬ 
bodied  in  current  configuration  management  regulatory  documents 
when  adapting  to  specific  program. 

-  Inadequate  SPO  planning  for  configuration  management. 

-  Inadequate  system/equipment  specifications;  inadequate  use  of 
specifications. 

-  Failure  to  establish  configuration  pre-baseline  control  and  docu¬ 
mentation  procedures. 

-  Failure  to  set  configuration  baseline  by  end  items. 

-  Failure  to  designate  proper  program  event  for  start  of  baseline 
control. 

-  Air  Force  acceptance  of  end  items  and  change  control  deferred 
until  turnover  of  total  system. 

-  Requirements  for  Air  Force  approval  limited  to  major  change  of 
engineering  significance  only,  up  to  the  point  of  turnover  of  first 
operational  system. 

•  Contractor  lacks  full  appreciation  of  Air  Force  configuration  manage¬ 
ment  objectives,  as  well  as  understanding  of  the  procedures;  where 
contract  work  statement  is  not  definitive  these  conditions  usually 
exist: 

-  Top  management  did  not  centralize  program  management,  and  is 
reluctant  to  do  so. 

-  Managed  according  to  self-determined  internal  requirements  based 
on  past  limited  role  as  equipment/vehicle  manufacturer. 

-  Underestimated  task  of  configuration  management  when  responsi¬ 
ble  for  producing  equipment  compatible  with  total  system  or 
responsible  for  integration  of  system. 

-  Adhered  to  conventional  semi-autonomous  line  organizations  with 
fragmented  change  control  and  administration. 

-  Policies,  procedures  and  documentation  inadequate  to  integrate 
change  control,  identification,  and  status  accounting  with  semi- 
autonomous  line  organizations. 

-  No  management  overview  of  the  complete  internal  configuration 
management  function  and  documentation  flow  between  engineer¬ 
ing  groups  and  between  engineering — manufacturing — material — 
test — product  services  or  site  activation  organizations. 
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•  Discontinuities  induced  by  engineering  practices. 

-  Semi-autonomous  design  groups,  each  with  independent  design 
release  authority  for  all  practical  purposes. 

-  Engineering  design  release  and  drawing  control  practices  inade¬ 
quate  for  integrated  system  design. 

-  Engineering  release,  for  a  given  design  deficiency,  does  not  package 
and  uniquely  identify  total  change  to  all  affected  equipment/ 
spares/procedures. 

-  Do  not  employ  packaged  change  control  with  multiple  engineering 
release  capability,  i.e.,  retain  in  engineering  data  bank  old  con¬ 
figuration  still  in  use,  along  with  new  configuration  which  is  a 
single  design  solution  of  the  problem  for  both  the  production  and 
the  field  retrofit  change. 

-  Design  parameters/detailed  criteria  for  design  change  levels  at 
which  system  analysis/review  is  mandatory  are  not  established 
definitively  or  set  at  proper  design  indenture  levels. 

-  System  design  review/analysis  is  not  prerequisite  for  any  design 
change  release. 

-  Lax  discipline  and  control  of  drawing/part  numbers;  engineering 
by-passes  formal  company  change  procedures  as  matter  of  general 
practice. 

•  Further  degradations  of  configuration  control,  identification  and 

status  accounting. 

-  Lack  of  a  central  change  control  board  with  representation  from 
all  internal  action  organizations. 

-  Inadequate  documentation  of  changes  during  the  manufacturing 
process. 

-  Inadequate  documentation,  status  accounting  and  maintenance  of 
equipment  log  books  by  company  field  organization. 

Actions  By  The  Contractor: 

•  Centralization  of  program  management. 

-  Established  central  program  configuration  management  and  ad¬ 
ministration  with  configuration  control  board. 

-  Revised  internal  configuration  management  policies  and  pro¬ 
cedures  to  conform  with  AFSCM  375-1  concepts. 

-  Revoked  authority  for  independent  release  by  design  groups. 

•  Established  packaged  change  control  procedures  with  multiple 

engineering  release  capability. 

•  Established  technical  criteria  for  mandatory  system  analysis/reviews. 

•  Design  review  group  formed  and  staffed  with  senior  design  specialists 

to  review  changes  from  system  viewpoint. 
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Actions  By  Air  Force: 

•  Numerous  Air  Force,  AFSC  and  joint  AFSC/AFLC  regulations  and 
manuals  as  well  as  military  specifications  pertaining  to  engineering 
changes  were  revised  to  reflect  the  new  policies  and  procedures  for 
configuration  management. 

-  Air  Force-Navy  Aeronautical  Bulletin  No.  445,  "Changes:  Engi¬ 
neering  to  Weapons,  Systems,  and  Equipments”  scheduled  for 
tri-service/industry  distribution  on  22  May  1963.  This  new  bulle¬ 
tin  combines  and  supersedes  ANA  Bulletins  390a  and  391a;  it 
revises  engineering  change  policies  and  procedures  so  that  they  are 
compatible  with  AFSC  Manual  375-1.  While  Bulletin  445  is 
AF-Navy  sponsored,  it  has  been  coordinated  with  and  will  be  used 
by  the  Army  also. 

•  AFSC  Manual  375-1,  "Configuration  Management  During  the 
Acquisition  Phase”,  was  published  1  June  1962.  This  manual  pro¬ 
vides  guidance  for  implementing  established  policies  on  configura¬ 
tion  management  by  delineating  responsibilities  and  defining  typical 
formats;  for  making  concurrent  decisions  for  approval  or  disapproval 
of  development,  production  and  retrofit  requirements  of  engineering 
changes  and  implementing  these  decisions;  and  for  configuration 
accounting  during  the  acquisition  phase  of  a  given  series  of  a  system/ 
equipment  program. 

•  The  established  objectives  of  configuration  management  are  to 
insure  that: 

-  All  end  items  of  equipment  in  a  given  series  will  be  the  same. 

-  Appropriate  procedures,  documentation  and  organizations  are 
initiated  and  completed  during  the  design,  development,  and  early 
test  phases  to  facilitate  transition  into  formal  configuration  man¬ 
agement. 

-  Delivered  end  items  of  equipment  are  accurately  and  completely 
described  by  all  identifying  documentation. 

-  For  every  approved  change  in  end  item  configuration,  the  cor¬ 
responding  changes  are  made  in  all  related  support  elements  of 
Aerospace  Ground  Equipment  (AGE),  including  depot  tooling, 
spare  parts,  training  equipment,  technical  order  data,  engineering 
data,  checkout,  operational  and  maintenance  computer  programs, 
and  attending  records. 

-  There  exists  a  configuration  record  documenting  all  approved 
changes  to  all  end  items. 

-  All  changes  involving  more  than  one  Air  Force  Configuration 
Control  Board  (CCB)  will  be  fully  coordinated  prior  to  implemen¬ 
tation. 

-  The  specific  location  and  status  of  each  end  item  of  equipment,  by 
part  number  and  serial  number,  are  known  at  all  times  during  the 
acquisition  phase. 

-  The  specific  location  and  status  of  end  items  of  equipment  that 
have  been  selected  as  configured  articles  are  known  by  part  num¬ 
ber  and  serial  number  at  all  times  during  the  operational  phase. 
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-  All  proposed  system/equipment/facility  changes  will  be  evaluated 
and  resolved  by  the  cognizant  AF-CCB. 

•  All  new  programs  entering  into  the  acquisition  are  required  to 
comply  with  AFSCM  375-1,  including  the  pre-baseline  configura¬ 
tion  management  procedures. 

•  All  programs  already  under  contract  and  in  the  acquisition  phase 
are  required  to  implement  the  procedures  of  AFSCM  375-1  as 
rapidly  as  the  training  of  SPO  people  permits  and  to  the  extent 
possible  within  constraints  of  the  given  program. 


Actions  To  Be  Accomplished  By  the  Air  Force: 

•  AFSC  is  developing  a  manual  for  systems  management.  This 
manual  will  include  policies  and  procedures  for  the  Program  Defini¬ 
tion  Phase.  AFSCM  375-1  will  be  revised  as  necessary  to  be  com¬ 
patible  with  this  new  manual. 


Additional  Actions  Recommended: 

•  Contractors  should  become  familiar  with  the  requirements  of 
AFSCM  375-1  if  they  are  not  now  acquainted  with  these  policies 
and  procedures. 

•  Contractors  practicing  decentralized  program  management  should 
review  and  revise  internal  policies  and  procedures  to  insure  integra¬ 
tion  of  total  program  activities,  for  which  they  are  responsible, 
throughout  all  functional  departments. 
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SUBJECT 

LATE  DEFIMTIZATION 
OF  LETTER 
CONTRACTS 


FINDING  No.  3 

DELAYS  IN  DEFINITIZING  LETTER  CONTRACTS  RESULT  IN 
CREATION  OF  WORK  FORCES  WITHOUT  POSITIVE  DIREC¬ 
TION,  HANDICAP  PROGRESS  EVALUATION,  STIMULATE 
CONTINUED  PROGRAM  RE-DIRECTION  AND  EXPENDITURE 
OF  AIR  FORCE  FUNDS  ON  TASKS  THAT  DO  NOT  CON¬ 
TRIBUTE  FULLY  TO  ACHIEVEMENT  OF  PROGRAM  OBJEC¬ 
TIVES. 


Symptoms  of  Basic  Deficiencies: 

•  Inability  of  Air  Force  and  contractor  to  reach  agreement  on  scope 
of  work  without  prolonged  negotiations. 

•  Cost  controls  and  work  keyed  to  contractor  estimates  rather  than 
firm  contract  provisions. 

•  Disagreement  between  Air  Force  and  contractor  on  cost  estimates. 

•  Work  nearly  completed  under  letter  contract. 


Fundamental  Causes: 

•  Work  statement  not  definitive  because: 

-  AF  requirements  not  adequately  defined,  or 

-  Contractor  analysis  of  AF  requirements  was  hasty  and  incomplete 
(Ref  System  Integration  Deficiency). 

-  Contractor  lacked  firm  procedures  for  developing  proposals. 

•  Disagreement  on  cost  estimates  because: 

-  Contractor  estimates  not  supported  by  definitized  statement  of 
work,  or 

-  Different  interpretation  of  AF  requirements. 

-  Contractor  estimates  not  developed  from  cost  data  accumulated 
from  similar  work  performed. 

•  Desire  to  keep  contract  open  so  that  new  work  can  be  added  with 

minimum  of  administrative  actions. 

•  Negotiations  unduly  prolonged  after  initial  disagreements  because: 

-  Delay  used  as  negotiation  tactic. 

-  Negotiators  are  often  reluctant  to  closely  question  methods  and 
techniques  used  by  the  technical  groups  that  developed  or  evalu¬ 
ated  technical  work  statements  and  cost  estimates. 

•  Desire  to  accumulate  actual  cost  data  before  agreeing  to  fixed  or 

target  prices. 

•  Slowness  in  negotiating  letter  subcontracts. 

•  Weak  AF  bargaining  position  when  going  to  sole  source. 
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Actions  By  Contractors: 

•  Revised  procedures  and  methods  of  developing  proposals  which 
emphasize  definition  of  work  and  cost  estimating. 

•  Education  and  training  programs  established  for  preparation  of 
proposals. 

Actions  By  Air  Force: 

•  Revised  Request  for  Proposal  checklist  and  instructions  to  obtain 
improved  statements  of  AF  requirements  (PMI  1-9). 

•  Training  program  set  up  to  develop  improved  cost  estimating 
methods  and  to  improve  in-house  AF  estimating  capability. 

•  Ballistic  Systems  Division  Exhibit  26-101  requires  more  detailed 
system  definition  requirements  for  missile  programs. 

•  Take  time  to  definitize  contract  at  the  outset. 

•  Buy  first  increment  of  work  when  impossible  to  define  entire 
program. 

•  AFSC  Controlled  Letter  Contract  Reduction  (CLCR)  Plan  estab¬ 
lishes  "number"  and  "dollar”  levels  for  each  division,  which  cannot 
be  exceeded  without  Hq  AFSC  approval. 

•  AFSC  emphasis  on  challenging  delivery  schedules  to  validate  essen¬ 
tiality  of  letter  contracts. 

•  AFSC  emphasis  on  technical/procurement  interface  to  improve 
completeness  and  timeliness  of  work  statements. 

Actions  To  Be  Accomplished: 

•  Program  definition  concept  being  developed  by  DOD  and  the  serv¬ 
ices  will  keep  two  or  more  competitors  active  until  definitive  con¬ 
tract  is  signed  with  one. 

•  Revise  AFSC  Program  Management  Instructions  to  require  more 
detailed  procurement  planning  in  Proposed  System  Package  Pro¬ 
grams  and  System  Package  Programs.  Emphasize  alternatives  to 
letter  contracts  and  definization  milestones  when  letter  contracts  are 
unavoidable. 
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SUBJECT 


FINDING  No.  4 


COST  CONTROL  AND 

ACCOUNTING 

DEFICIENCIES 


INADEQUATE  COST  CONTROL  AND  ACCOUNTING  PRAC¬ 
TICES  CONTRIBUTED  TO  INACCURATE  COST  ESTIMATES, 
UNANTICIPATED  OVERRUNS  AND  DELAYED  PROGRAM 
DECISIONS. 


Symptoms  of  Basic  Deficiencies: 

Unable  to  identify  and  compare  detail  actual  costs  with  proposals. 

•  Improper  identification  and  allocation  of  costs  to  contracts. 

•  Cost  proposals  incomplete. 

•  Late  cost  proposals. 

•  Cost  proposals  included  unrelated  elements. 

•  Delays  in  contract  negotiations  and  definitization. 

•  Potential  overruns  identified  late  to  AF  management. 

Fundamental  Causes: 

•  Inadequate  segregation  of  contract  costs. 

•  Systems  do  not  allocate  costs  accurately  on  multiple  contracts  for 
same  production  items. 

•  Cost  controls  do  not  relate  work  accomplishment  to  actual  costs. 

•  Inadequate  or  nonexistent  records  and  management  information 
about  business  data  processing. 

•  Costs  not  accumulated  against  work  packages. 

•  Air  Force  changing  data  requirements  cause  confusion  within  the 
accounting  system. 

•  Air  Force  not  gaining  maximum  benefit  from  contractor  manage¬ 
ment  experience. 

-  Air  Force  denied  access  to  records. 

-  Cost  data  not  effectively  utilized  for  contract  program  control 
purposes. 

•  No  formal,  centralized  estimating  system. 

•  Program  manager  not  familiar  with  cost  details. 

-  No  system  of  controlling  costs  of  work  transferred  to  another 
division. 

-  Gross  fund  allocations  to  operating  divisions  with  no  requirement 
for  regular  expenditure  reports. 
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Actions  By  Contractors: 

•  Program  to  instill  cost  consciousness  in  middle  management. 

.  •  Revised  accounting  and  estimating  system  to  correct  co-mingling 
of  direct  charges. 

•  Instituted  a  program  to  develop  and  direct  a  "Cost  Proposal 
System.” 

•  Established  a  Program  Review  Board  to  refine  cost  control,  budget¬ 
ing  and  cost  reporting  systems. 

•  Piepared  a  cost  estimating  manual. 

•  Appointed  cost  control  coordinator. 

•  Planned  expenditure  rates  established  and  monitored  on  all  major 
contracts. 

•  Established  system  to  provide  cost  information  by  sub-project  or 
work  package. 

•  Updated  procedure  manuals  and  improved  forms  control. 

•  Developed  EDP  system  for  maintaining  up-to-date  contract  value  for 
all  authorized  work  and  a  budgetary  estimate  for  un-negotiated 
authorized  work. 

•  Established  dollar  and  quantity  controls  on  overhead  supplies. 

•  Reduced  volume  and  complexity  of  cost  transfers. 

•  Enforced  cost  transfer  pricing  directives. 

Actions  By  Air  Force: 

•  Verify  that  contractors  use  current  historical  data. 

•  Improved  contractor  cost  estimating  manual. 

•  Revised  AFSC  "Contractor  Cost  Study”  Program. 

•  Revised  AFSC  pricing  manual. 

•  PERT/Cost  test  program. 

•  AFSC  plan  for  a  source  of  historical  cost  information  for  use  by 
all  AFSC  organizations. 

•  Cost  Estimating  Course  established  at  Wright-Patterson  AFB  to 
train  AF  personnel  in  estimating  and  analysis  capability. 

•  RAND  Computer  Program  for  total  force  structure  costs  for  ten 
years.  Costs  developed  by  system,  mission  and  support,  include  total 
resources  allocated  yearly  to  the  Air  Force  based  on  program  re¬ 
quirements,  with  shred-outs  by  mission,  weapon  system,  groups  of 
systems  and  R&D;  investment  and  operating  costs  by  appropriations. 

•  Use  of  “General  Standard  for  System  Source  Selection  Cost  Pro¬ 
posals”  to  stimulate  submission  of  proposals  in  similar  format  and 
detail,  consistent  with  AFSC  Contractor  Cost  Study. 
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SUBJECT 

MAKE-OR-BUY 
POLICIES  NOT 
ENFORCED 


FINDING  No.  5 

MAKE-OR-BUY  DECISIONS  WERE  NOT  MADE  OR  EVALU¬ 
ATED  IN  ACCORDANCE  WITH  ASPR  POLICY  OR  INTENT, 
THEREBY  PERMITTING  POOR  UTILIZATION  OF  INDUS¬ 
TRIAL  RESOURCES,  CONTRIBUTING  TO  LATE  DELIVERIES, 
POOR  PERFORMANCE,  AND  INCREASED  COSTS. 

Symptoms  of  Basic  Deficiencies: 

•  Contractor  making  items  not  normal  to  production. 

•  Absence  of  competitive  procurement  by  contractors. 

•  Domination  of  make-or-buy  decisions  by  one  department. 

•  Assignment  of  potentially  competitive  fixed-price  buy  items  to  other 
corporate  divisions  on  cost  reimbursement  work  orders. 

•  Pulling  in  large  dollar-value  subcontracts  without  economic  justi¬ 
fication. 

•  Expansion  of  facilities  or  work  force  when  adequate  capacity  exists 
elsewhere. 

•  Little  or  no  documentation  of  "make”  decisions. 

•  Establishment  of  arbitrary  make-or-buy  ratios. 

•  Protracted  AF  review  of  contractor  proposals. 

Fundamental  Causes: 

•  Contractor  directives  do  not  provide  specific  guidance. 

•  No  corporate  enforcement  of  directives  on  operating  divisions. 

-  Directives  confined  to  existing  make-or-buy  structures. 

•  Little  or  no  analysis  by  contractor  or  AF  buyers. 

-  Contractors  do  not  evaluate  all  factors  in  developing  a  make-or- 
buy  program. 

-  Purchasing  contracting  officers  rubber  stamping  make-or-buy 
structures. 

-  Insufficient  evaluation  of  make-or-buy  proposals  by  AF  field 
activities. 

•  Letter  contracts  signed  prior  to  review  or  approval  of  make-or-buy 
proposals. 

-  Contractor  later  negotiates  from  "fact  accomplishment”  position. 

•  Undue  emphasis  on  one  or  two  factors  in  evaluation  by  AF  buying 
and  field  activities. 
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Actions  By  Contractors: 

•  Internal  audit  of  make-or-buy  compliance. 

•  Revising  instructions  to  incorporate  the  ASPR  and  AFPI  require¬ 
ments. 

•  Detailed  operating  procedures  to  provide  guidance  and  direction 
on  how  data  used  in  the  make-or-buy  determination  will  be  ob¬ 
tained. 

•  Document  rationale  for  decisions. 

•  Establish  make-or-buy  committee. 

Actions  By  Air  Force: 

•  AFSC  study  group  analyzing  make-or-buy  problems  in  defense  in¬ 
dustry. 

•  AFSC  action  to  standardize  contractor  data  in  support  of  make-or- 
buy  programs  to  the  minimum  consistent  with  the  intent  of  ASPR. 

•  Division  supplements  to  ASPR  requiring  early  submission  of  pro¬ 
posed  make-or-buy  structure. 

Actions  To  Be  Accomplished: 

•  More  fixed-price  and  incentive  contracts  which  obviate  government 
concern  with  contractors’  make-or-buy  decisions  (unless  use  of 
government-owned  facilities  is  involved). 
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SUBJECT 

COSTS  ARE 
UNDERESTIMATED 


FINDING  No.  6 

CONSISTENT  UNDERESTIMATING  OF  COSTS  CAUSES  DE¬ 
LAYS  IN  PROGRAM  DECISIONS,  PROGRAM  STRETCH-OUTS 
OR  TERMINATIONS,  AND  RESULTS  IN  COST  OVERRUNS. 

Symptoms  of  Basic  Deficiencies: 

•  Consistent  underestimates  of  engineering  hours,  particularly  on 
cost  reimbursement  contracts. 

•  Prolonged  delays  in  negotiations  and  in  definitizing  letter  contracts. 

•  Support  data  not  developed. 

•  Use  of  questionable  and  obsolete  estimating  factors. 

•  Estimating  systems  and  procedures  poorly  defined. 

•  Estimates  overdue. 

Fundamental  Causes: 

•  Incomplete  description  of  products  procured. 

•  Lack  of  management  attention. 

•  Failure  of  AF  to  emphasize  procedures. 

•  The  AF  and  contractors  had  not  reached  a  mutual  understanding 
as  to  what  constituted  an  acceptable  estimating  system. 

•  Inability  to  support  estimating  factors. 

•  Insufficient  time  allowed  (or  available)  to  prepsure  realistic  estimates. 

Actions  By  Contractors: 

•  Regularly  reviewing  estimating  factors. 

•  Accepting  the  bilateral  (AF-contractor)  method  of  improving  esti¬ 
mating  methods. 

•  Develop/update  estimating  manual. 

•  Establish  estimating  training  program. 

Actions  By  Air  Force: 

•  Review  and  revise  estimating  factors  if  necessary  every  six  months. 

•  Reviews  of  major  contractors’  cost  estimating  techniques  on  con¬ 
tinuing  basis. 
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•  Published  AFSCM  70-4,  20  Dec  62,  for  review  of  contractor  estimat¬ 
ing  methods.  Will  provide  unified  guidance  and  improve  estimating 
methods  review  procedures. 

•  Evaluation  of  outstanding  contractor  estimating  techniques  which 
may  be  useful  throughout  defense  industry. 

•  Ballistic  Missile  Division  Exhibit  62-101  on  program  definition 
being  service  tested  on  MMRBM  and  Titan  Ill.  AFSC  working 
groups  are  preparing  a  system  management  manual  which  will  have 
a  publication  date  established  in  July  1963.  This  manual  will  in¬ 
clude  Program  Definition  instructions. 
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SUBJECT 

INEFFECTIVE 
MANAGEMENT  AUDIT 


FINDING  No.  7 

CONTRACTORS  GENERALLY  WERE  NOT  ASSESSING  EFFEC¬ 
TIVENESS  OF  OR  COMPLIANCE  WITH  COMPANY  POLICIES 
AND  PROCEDURES. 


Symptoms  of  Basic  Deficiencies: 

•  Policies  and  procedures  out  of  date. 

•  Conflicting  directives  assign  responsibilities  to  more  than  one  de¬ 
partment  or  division. 

•  Duplication  of  work  by  separate  departments. 

•  Confusion  generated  from  failure  to  follow  established  directives. 

•  No  written  procedures  to  cover  accepted  standard  practices,  with 
consequent  problem  of  training  new  employees. 

•  Staff  assumption  of  line  responsibility. 

•  Work  initiated  without  authorization. 

•  Basic  management  decisions  requiring  action  by  several  departments 
not  documented  or  publicized. 

•  Unfamiliarity  of  program  managers  with  supporting  departmental 
procedures. 

•  Undefined  areas  of  program  management  responsibility. 

•  Prolonged  communication  processes. 

(Surveys  indicate  a  direct  relationship  between  the  number  and 
seriousness  of  basic  deficiencies  and  the  effective  use  of  management 
audits.) 


Fundamental  Causes: 

•  Lack  of  acceptance  and  recognition  for  the  need  of  independent 
policies  and  procedures  audit  system. 

•  Fragmentary  management  audits  without  comprehensive  plan. 

•  Audit  limited  to  financial  activities. 

•  Understaffing  of  audit  activity. 

•  Audit  group  reported  to  operational  activity  instead  of  top  manage¬ 
ment. 

•  Weak  or  no  follow-up  system. 

Actions  By  Contractors: 

•  Establishment  of  permanent  independent  audit  groups,  reporting  to 
top  management. 
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•  Establishment  of  corporate  audit  cadre  to  form  ad  hoc  surveys  of 
division  program  management. 

•  Establish  top  management  ad  hoc  audit  groups. 

Actions  By  Air  Force: 

•  Coordination  between  government  and  company  audit  activities  to 
reduce  duplication. 

•  Encourage  contractors  to  establish  independent  internal  policy  and 
procedures  audit  and  reduce  need  for  government  inspection. 

•  Establishment  of  AFSC  management  survey  activities. 
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SUBJECT 

LIMITED  STANDARDS 
FOR  MANPOWER 
UTILIZATION 


FINDING  No.  8 

STANDARDS  FOR  ASSESSING  MANPOWER  UTILIZATION/ 
REQUIREMENTS  GENERALLY  WERE  RUDIMENTARY  AND 
APPLIED  TO  A  LIMITED  PORTION  OF  THE  WORK  FORCE. 
AS  A  RESULT  MANAGEMENT  EXERCISED  ONLY  LOOSE  CON¬ 
TROL  OVER  THE  LARGEST  SINGLE  ELEMENT  OF  COST. 


Symptoms  of  Basic  Deficiencies: 

•  Manpower  controls  ere  based  on  historic  employment  data  and  gross 
estimates  of  people  needed  for  projected  tasks.  Such  methods  in* 
corporate  organizational,  procedural,  and  supervisory  inefficiencies 
in  the  historical  base. 

•  Little  or  no  effort  to  base  standards  for  unit  or  individual  perform¬ 
ance  on  actual  work  measurement. 

•  Use  of  handbook  "feeds  and  speeds”  data  rather  than  timed  opera¬ 
tions. 

•  Use  of  "activity  indicator”  work  measurement  systems  not  related  to 
actual  productivity. 

•  No  standards  or  guides  for  supervisor — worker  ratios. 

•  No  standard  hours  or  cost  data. 


Fundamental  Causes: 

•  Cost  reimbursement  contracts  do  not  motivate  companies  to  manage 
manpower  efficiently. 

•  Preference  of  buying  activities  for  contractors  with  "in  being" 
engineering  capabilities. 

•  Resistance  of  departmental  managers  to  have  performance  reported 
as  a  numerical  rating. 

•  Preference  of  middle  managers  to  justify  overall  manpower  ceilings 
rather  than  match  jobs  against  specific  measured  standards  which 
take  out  "fat.” 


Actions  By  Contractors: 

•  Established  work  measurement  programs  in  direct  labor  and  support¬ 
ing  departments. 

•  Evaluate  departmental  "earned  hours”  against  standard  hours  with 
consequent  improvement  in  efficiency  and  reduction  of  labor  force. 

•  Established  manpower  authorized/controls  based  on  job  measure¬ 
ment  and  task  definition. 
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Actions  By  Air  Force: 

•  Emphasizing  fixed  price  and  incentive  contracts  which  will  motivate 
contractors  to  develop  and  apply  manpower  controls. 

•  Malting  manpower  controls  special  subject  of  management  surveys. 
Actions  Recommended: 

•  Rate  contractors’  manpower  controls  as  major  management  factor  in 
source  selection. 
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SUBJECT 

DEFICIENT  REPORTS 
CONTROL  PROCEDURES 


FINDING  No.  9 

FAILURE  TO  CENTRALLY  MANAGE  REPORTS  RESULTED  IN 
PROLIFERATION  AND  DUPLICATION  WITH  CONSEQUENT 
UNNECESSARY  COST  INCREASES. 

Symptoms  of  Basic  Deficiencies: 

•  No  central  review  or  control  of  internal  or  external  reporting  re¬ 
quirements. 

•  Duplication/overlapping  of  reports. 

•  No  formal  reports  control  program. 

•  No  current  inventory  of  reports. 

•  No  accounting  of  individual  report  costs. 

•  No  periodic  evaluation  of  individual  report  need  and  use  versus  cost. 

•  No  central  forms  control  activity. 

Fundamental  Causes: 

•  Reports  producing  group  functions  as  service  organization  with  no 
responsibility  to  make  management  evaluations. 

•  Management  not  aware  of  manhour  or  dollar  costs  of  total  reports, 
or  extent  of  duplication. 

Actions  By  Contractors: 

•  Established  central  reports/forms  control  group. 

•  Approval  mandatory  for  new  reports. 

•  Up-dated  complete  report  inventory. 

•  Altered  accounting  procedures  to  produce  costs  of  individual  reports. 

•  Established  formal  reports  control/reduction  program. 

•  Require  periodic  economic  justification  of  existing  reports  and  tab 
runs. 


Actions  By  Air  Force: 

•  Reduced  contractual  reporting  requirements. 

•  Standardized  procedures  (PMI 1-8). 

•  Centralized  control  (SPO)  of  contractual  reporting  requirements. 

•  Periodic  reviews — AF/Industry  on  specific  weapon  system  programs. 
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SECTION  II 

Engineering  Functions 

Finding  No.  Page  No. 

10.  Insufficient  System  Design  and  Detailed  Design  Integration .  33 

11.  Inadequate  Consideration  of  Reliability  in  Detailed  Design;  Inadequate 

Utilization  of  F&CD  by  Design  Groups .  39 

12.  Unrealistic  Development  Engineering  and  Test  Schedules .  43 

13.  Inefficient  use  of  Engineering  Manpower,  and  Assignment  and  Control 

of  Engineering  Work .  45 
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SUBJECT 

INSUFFICIENT  SYSTEM 
DESIGN  AND  DETAILED 
DESIGN  INTEGRATION 


FINDING  No.  10 

SOME  DEGREE  OF  DIFFICULTY  WAS  EXPERIENCED  BY  ALL 
CONTRACTORS  IN  THESE  TWO  TECHNICAL  MANAGEMENT 
AREAS.  A  WIDE  VARIETY  OF  AFTEREFFECTS  CAN  BE 
TRACED  TO  THE  ENGINEERING  PRACTICES  USED  IN  THE 
INITIAL  SYSTEM  DESIGN  PROCESS.  OTHER  EFFECTS  ARE 
CAUSED  BY  PRACTICES  USED  IN  THE  DETAILED  DESIGN 
PHASE  WHEREBY  EXCESSIVE  ANOMALIES  ARE  INTRO¬ 
DUCED  INTO  THE  TOTAL  SYSTEM.  IN  GENERAL,  THE 
QUALITY  AND  DEPTH  OF  ANALYTICAL  WORK  THAT  IS  A 
FUNDAMENTAL  PART  OF  THE  EQUIPMENT  DETAILED  DE¬ 
SIGN  PROCESS  DOES  NOT  COMPARE  FAVORABLY  WITH 
THAT  AT  THE  SYSTEM  DESIGN  LEVEL.  THE  LIMITED 
SCOPE  OF  THE  EQUIPMENT  DESIGNER'S  EFFORT  IS 
GRAPHICALLY  REVEALED,  BELATEDLY,  WHEN  THE  HARD- 
WARE  IS  PHYSICALLY  INTEGRATED  INTO  THE  SYSTEM. 


Symptoms  of  Basic  Deficiencies: 

•  High  percentage  of  design  changes  are  made  to  correct  previous 
design  oversights. 

•  Letter  contracts  in  force  for  prolonged  periods — as  long  as  two  or 
three  years. 

•  Subcontract  and  in-house  workstatements  not  definitive,  usually 
attended  by  inability  to  relate  engineering  hours  expended  to  design 
accomplished  and  the  occurrence  of  contract  cost  overruns. 

•  Identification  and  justification  of  AGE  and  facility  items  late,  in¬ 
complete  and  inadequate. 

•  Development  of  system/equipment  interface  specifications  and  de¬ 
tailed  specifications  late  and  inadequate  to  needs. 

•  Development  of  technical  data  late  and  of  poor  quality. 

•  High  rate  of  system  design  incompatibilities  revealed  by  system  test¬ 
ing;  feedback  from  system  test  inefficiently  utilized  or  given  low 
priority  treatment  by  design  groups. 

•  Essential  reliability  considerations  are  predominantly  made  on  after- 
the-fact  basis,  i.e.,  after  design  release. 

•  Unilateral  design  releases  by  de«'  —  groups  (group  release)  permitted 
and  constitute  highest  percentage  of  total  release  design. 

•  Engineering  recommendations  or  proposals  not  supported  by  analy¬ 
sis;  potential  design  trade-offs  not  identified. 

Fundamental  Causes: 

•  Air  Force  and  contractor  top  management  are  prone  to  apply  cor¬ 
rections  to  secondary  effects  or  visible  problems  rather  than  to  seek 
and  correct  basic  causes;  e.g.,  joint  negotiation  team  is  set  up  to 
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definitize  letter  contract  when  cause  of  late  definitization  is  an 
inadequate  engineering  or  technical  workstatement  wherein  AF/ 
contractor  engineering  groups  assert  better  definitization  not  possible 
until  system/equipment  is  invented — at  this  point,  top  management 
does  not  question  whether  engineering  has  applied  adequate  methods 
and  techniques  in  developing  workstatement,  or  has  simply  made 
off-the-cuff  "guettimates”. 

•  In  system  acquisition  programs — top  management  has  often  failed 
to  appreciate  the  extent  to  which  the  engineering  discipline  should 
be  the  forcing  function  in  the  acquisition  process  and  as  a  corollary, 
they  do  not  recognize  the  magnitude  of  the  cascade  effect  of  inade¬ 
quate  and  incomplete  engineering  upon  production,  purchasing, 
materiel  control,  quality  control,  logistics,  etc.,  and  ultimately  the 
impact  upon  the  customer. 

•  Many  contractor  engineering  top  managers,  as  they  have  advanced 
to  present  positions,  have  retained  biases  of  the  design  engineer 
which  are  reflected  in  the  organization  and  management  of  the  en¬ 
gineering  effort — propensity  for  undue  personal  involvement  with 
the  more  interesting  and  esoteric  design  problems — tendency  to 
organize  solely  for  hardware  design  task  with  slight  attention  to 
equally  fundamental  responsibilities  for  output  of  technical  data  and 
timely  support  of  other  departments — tendency  toward  excessive 
"management  by  exception”  where  organizationally  and  procedurally 
they  have  not  adequately  prescribed  realistic  technical  and  manage¬ 
ment  boundary  condi tit  -.s  or  criteria  that  will  insure  control,  which 
in  turn  results  in  significant  technical  decisions  being  made  at  or  be¬ 
low  the  group  engineer  levels  that  are  unknown  to  higher  manage¬ 
ment  or  where  important  engineering  functions  are  unaccomplished 
and  not  detected  until  highlighted  as  a  problem. 

•  The  following  factors  predominate  in  producing  cascades  of  prob¬ 
lems  in  contract  definition,  purchase,  cost  control,  materiel  control, 
production,  quality  control,  reliability,  system  check-out  and  turn¬ 
over  to  Air  Force.  The  conditions  noted  above  are  not  necessarily 
prerequisite  to  these  factors;  however,  the  impact  of  the  following 
factors  will  be  more  severe  if  those  conditions  exist. 

•  Simultaneity  of  the  design  process  which  is  an  inherent  character¬ 
istic  of  this  process  and  independent  of  whether  or  not  the  program 
concept  is  one  of  concurrency  or  of  sequenced  development,  pro¬ 
duction  and  deployment. 

-  Extreme  difficulty  of  timely,  accurate  communication  between 
large  numbers  of  designers  who  have  been  educated  and  specialize 
in  different  engineering  disciplines  and  fields  of  technology. 

-  Further  accentuated  by  the  iterative  nature  of  the  design  process. 

•  Initial  system  definition  either  was  not  acomplished  hy  an  orderly 
analysis  process  or  effort  was  incomplete  and  inadequate;  there  was 
no  continuing  requirement  to  perform  system  analysis  on  selective 
basis  during  acquisition  phase. 

•  Decentralized  program  management. 

-  Authority  to  make  design  decisions  and  design  releases  delegated 
to  too  low  organizational  levels. 
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-  Autonomous  or  semi-autonomous  design  groups  combined  with 
excessive  reliance  on  voluntary  lateral  coordination  between 
groups. 

-  Design  groups  predominantly  equipment  oriented  and  have  little 
capability  for  system  engineering  and  integration. 

-  Lack  of  design  review  for  system  compatibility  and  optimization 
above  group  engineer  levels. 

•  System  engineering  or  design  integration  group,  if  it  exists,  is  im¬ 
potent  because: 

-  Authority,  responsibility  and  accountability  not  clearly  defined,  or 

-  Group  has  limited  function  such  as  review  of  design  released  in 
administrative  sense  only  and  not  from  technical  viewpoint,  or 

-  Useful  criteria  for  determining,  routinely,  when  design  reviews 
or  system  analyses  will  be  performed  have  not  been  developed; 
or,  the  criteria  do  not  adequately  define  technical  conditions  in 
quantitative  terms  for  which  design  reviews  and  system  analyses 
are  mandatory  in  initial  design  and  design  changes. 

-  System  analyses  and  design  reviews  are  accomplished  only  upon 
request  of  design  groups. 

-  Design  Integration  Group  not  in  design  release  channel;  in  prac¬ 
tice  decisions  for  release  are  made  routinely  and  unilaterally  by 
design  groups. 

-  System  Engineering  or  Design  Integration  Group  lacks  technical 
competence  and  capability  to  perform  functions. 

•  Separate  department  external  to  engineering  established  to  manage 
reliability  program,  but  assigned  responsibilities  that  properly  can¬ 
not  be  separated  from  the  basic  design  function. 

Actions  By  Contractors: 

•  Voluntary  establishment  of  distinct  System  Integration  or  Design 
Review  Group,  as  appropriate,  with  functions  and  responsibilities 
clearly  defined  and  executed  by  experienced,  competent  senior  engi¬ 
neering  specialists. 

•  Reorganization  along  project  lines — vertical  organization  for  im¬ 
proved  program  control  and  design  decision  making. 

•  Education  and  training  of  personnel  in  preparation  of  engineering 
workstatements  and  engineering  change  proposals. 

•  Contractor  management  teams  with  buyer-engineer-materiel-quality 
control  representation  established  for  each  major  subcontract. 

•  Responsibility  and  accountability  for  reliability  considerations  in 
original  design  and  in  design  changes  reassigned  to  basic  design 
groups.  The  external  Reliability  and  Quality  Assurance  Department 
assigned  independent,  reliability  evaluations  and  manages  the  failure 
and  consumption  data  program,  testing  for  reliability  (search  for 
critical  weaknesses),  etc. 
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Actions  By  Air  Force: 

•  Request  for  proposals  (RFP)  now  includes  specific  instructions  on 
contract  and  program  definition  requirements. 

•  Increasing  use  of  two  or  three  phase  contracting  for  system  acquisi¬ 
tion  where:  Phase  I — is  solely  for  program  defiinition — a  funded 
effort  where  two  or  more  contractors  are  selected  on  competitive 
basis  to  perform  program  definition;  based  on  evaluation  their 
efforts  one  contractor  is  selected  for  a  Phase  II — using  the  program 
definition  developed  in  Phase  I,  Phase  II  may  be  either  for  RDT&E 
only  or  it  may  proceed  with  both  RDT&E  and  production. 

•  For  all  new  system  programs  in  the  future,  system  analysis  methods 
will  be  required  by  contract  to  be  employed  for  system  definition  and 
selectively  required  throughout  the  acquisition  phase  (example — 
BSD  Exhibit  62-101).  The  System  Analysis  Program  provides  the 
management  means  for: 

-  Identification  of  system  requirements  in  a  systematic  manner  on 
total  system  basis,  including  hardware,  procedural  data,  facilities 
and  personnel. 

-  Development  of  design  requirements  on  basis  of  time,  perform¬ 
ance  and  cost  trade-offs. 

-  Integration  and  trade-offs  of  design  constraints,  i.e.,  reliability, 
maintainability,  producibility,  procurability,  safety,  human  en¬ 
gineering  and  value  engineering. 

-  Progressive  and  evolutionary  development  and  approval  of  speci¬ 
fications  as  the  forcing  function  on  design. 

-  Establishment  of  method  of  control  which  permits  design  flexi¬ 
bility. 

-  Development  of  definitized  information  which  forms  basis  for  and 
inter-relates  test,  production,  system  activation  and  check-out. 

-  Provide  early  information  necessary  for  determination  of  items 
in  DOD  inventory  which  meet  system  requirement. 

-  Provide  basic  data  necessary  for  networking  system  milestone 
activities  into  program  review  control  techniques  such  as  PERT 
and  PERT/Cost. 

-  Provide  single  source  of  documentation  for  evaluating  design  to 
insure  total  system  objectives  are  met. 

•  For  older  programs  already  on  contract,  the  system  analysis  tech¬ 
niques  are  selectively  applied  by  contract  change  notices  (CCNs)  as 
required  to  produce  progressive  controls  of  Air  Force  requirements 
and  the  design  process  through  the  use  of  multiple  sequential  base¬ 
lines.  These  baselines  are  progressively: 

-  Design  requirements. 

-  Design  configuration. 

-  Hardware  configuration. 

•  AFPROs  or  CMDs  and  SPOs  follow-up  on  contractor’s  corrective 
actions  and/or  execution  of  the  contractual  requirements  for  system 
analysis  and  the  contractor’s  performance  in  design  integration. 
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Actions  To  Be  Accomplished  By  Air  Force: 

•  AFSC  is  developing  a  manual  for  Systems  Management  which  in¬ 
cludes  program  definition  instructions  and  system  analysis  procedures. 
This  manual  will  include  contract  exhibits  such  as  the  BSD  Exhibit 
62-101,  "System  Definition/Analysis  Procedures,”  which  will  be 
revised  to  reflect  the  practical  experience  gained  from  use  by  con¬ 
tractors  with  system  integration  and  sub-system/equipment  respon¬ 
sibilities. 


Additional  Actions  Recommended: 

•  Contractor  technical  managers  in  general  should  give  more  thought 
and  attention  to  their  total  management  role.  Periodically  they 
should  personally  check  into  specific  details  of  the  management 
system  to  determine  whether  the  system  is  functioning  in  practice 
as  they  had  believed.  Specifically,  they  should  have  more  intimate 
knowledge  of  the  actual  functional  flow  channels,  the  decision 
making  channels  and  levels,  the  adequacy  of  methods  and  techniques 
for  work  definition,  and  the  practices  in  work  approval,  assignment, 
and  control  for  items  not  routinely  brought  to  top  management 
levels. 

•  Critical  evaluations  should  be  made  by  the  Air  Force  and  contrac¬ 
tors  in  the  early  design  stages  concerning  the  specification  require¬ 
ments.  They  should  be  evaluated  from  both  viewpoints — too  tight, 
too  loose. 


s 


37 


1  June  1963 


AFSCP  375-2 


SUBJECT 

INADEQUATE 
CONSIDERATION  OF 
RELIABILITY  IN 
DETAILED  DESIGN; 
UTILIZATION  OF 
FAILURE  AND 
CONSUMPTION  DATA 
BY  DESIGN  GROUPS 


FINDING  No.  11 

THE  CONTRACTORS  WERE  GIVING  INADEQUATE  ATTEN¬ 
TION  TO  RELIABILITY  IN  INITIAL  DESIGN  EFFORTS  AND / 
OR  INSUFFICIENT  ATTENTION  LATER  ON  TO  FAILURE  AND 
CONSUMPTION  DATA  REPORTS. 


Symptoms  of  Basic  Deficiencies: 

•  Many  changes  to  delivered  equipment  are  required  to  obtain  mini¬ 
mum  acceptable  reliability,  with  attendant  higher  program  costs. 

•  High  spares  levels  to  support  development  tests  and/or  operational 
system. 

•  Upgrading  to  meet  approved  reliability  objectives  accomplished 
by  "follow-on”  programs. 

•  Contractor  design  groups  not  aware  of  failure  and  consumption 
trends  in  functional  and  development  tests  on  equipment  of  their 
design. 

•  Low  "in-commission”  status  of  equipment  in  custody  of  user  organi¬ 
zation;  using  agency  has  low  degree  confidence  in  reliability  of 
equipment. 


Fundamental  Causes: 

•  Air  Force  did  not  provide  quantitative  reliability  requirements 

timely  to  contractor’s  initial  design  efforts.  (Reference  program 

definition  and  detailed  design  integration  in  technical  section.) 

-  Failure  to  require  formal  program  definition  analyses  which 
would  have  highlighted  lack  of  requirements. 

-  Air  Force  planning  documents  (SOR,  PSPP,  and  BPP)  lacked 
quantitative  reliability  requirements. 

-  Lack  of  appreciation  and  understanding  that  quantitative  re¬ 
quirements  are  necessary  in  detailed  design  initial  efforts. 

-  Reluctance  to  place  quantitative  reliability  requirements  on  con¬ 
tract. 

-  Value  engineering  requirements  conflict  with  or  contradict  reli¬ 
ability  goals. 

-  Extreme  emphasis  on  program  cost  and  schedules  which  adversely 
affected  consideration  of  reliability  in  necessary  detail  during  the 
design  process. 
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*  Lack  of  full  appreciation  by  a  few  contractors  and  Air  Force  Pro¬ 
gram  Offices  that  consideration  of  reliability  is  integral  to  the 
design  process. 

-  Design  groups  were  not  held  responsible  and  accountable  for 
reliability  in  detail  design  because  complete  responsibility  for  all 
aspects  of  the  reliability  program  was  assigned  to  an  autonomous 
reliability  organization,  separate  and  apart  from  the  engineering 
department  and  which  was  ineffective  in  keeping  abreast  of  the 
detail  design  process.  (NOTE:  This  is  not  to  say  that  autonomous 
reliability  organizations  are  undesirable.  Such  organizations  are 
particularly  effective  where  this  function  is  one  of  reliability 
assurance,  i.e.,  management  of  the  reliability  data  reporting  sys¬ 
tem,  reliability  testing  of  components,  search  for  critical  weakness 
testing,  reliability  trend  analyses,  etc.) 

•  Inadequacies  of  the  Failure  and  Consumption  Reporting  System 
was  frequently  the  reason  for  apparent  failure  of  design  groups  to 
utilize  Failure  and  Consumption  Data. 

-  Failure  and  Consumption  Reports  did  not  contain  meaningful  or 
pertinent  information,  either  because  the  data  card  format  was 
not  sufficiently  comprehensive,  or  field  personnel  lacked  proper 
training  in  making  card  entries. 

-  Excessive  flow  times — 4  to  6  months — for  Failure  and  Consump¬ 
tion  Data  Reports  to  filter  in  from  field  through  the  reliability 
organization  to  the  design  groups.  (However,  design  groups  were 
usually  acting  on  information  from  field  obtained  directly  through 
engineering  channels  and  the  problem  usually  was  one  of  recon¬ 
ciling  that  anion  with  what  was  apparently  required  by  Failure 
and  Consumption  Data  Reports.) 


Actions  By  Contractors: 

•  Issued  new  policy  clarifying  responsibilities  of  engineering  depart¬ 
ment  and  independent  reliability  organization  for  the  reliability 
program. 

-  Engineering  held  responsible  and  accountable  for  attainment  of 
reliability  program  design  objectives  or  requirements. 

-  Reliability  organization  redesignated  Reliability  Assurance  (or 
Quality  and  Reliability  Assurance)  Department  responsible  for 
management  of  the  reliability  data  gathering  system,  for  reli¬ 
ability  trend  analysis,  testing  for  reliability  of  components  and 
search  for  critical  weaknesses,  functioning  as  the  policing  agency 
to  assure  that  the  reliability  loop  is  closed  throughout  the  re¬ 
quirements,  design,  development,  production  and  deployment 
cycle. 

•  Several  companies  have  analyzed  their  entire  design  process  to  im¬ 
prove  methods  and  techniques  of  designing  and  to  develop  a  better 
capability  to  "design  it  right  the  first  time.” 

-  The  increased  use  of  high-speed  digital  computers,  statistical 
theory,  8nd  mathematical  models  in  the  initial  design  process. 
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•  Revised  Failure  and  Consumption  Data  Card  formats  in  order  to 
obtain  more  comprehensive  data;  educated  field  personnel  in  re¬ 
liability  data  reporting. 

•  Streamlined  Failure  and  Consumption  data  reporting  channels  to 
provide  timely  information  and  engineering. 

Actions  By  Air  Force: 

•  System  Acquisition  Programs  are  now  required  to  undergo  a  Pro¬ 
gram  Definition  Phase.  One  objective  of  this  phase  is  the  early 
identification  of  total  reliability  requirements  for  injection  into 
the  original  designs. 

•  Military  Specification  MIL-R-27542  and  AFSC  Regulation  80-1  have 
been  updated  to  take  advantage  of  the  improved  reliability  tech¬ 
niques. 

•  AFSC  procedures  have  been  tightened  up  to  insure  that  quantitative 
reliability  requirements  are  both  in  the  planning  documents  and, 
later,  in  contracts  before  award  approval. 

•  Air  Force  Institute  of  Technology  has  established  several  formal 
courses  on  reliability,  one  of  which  leads  to  a  Masters  degree  in 
reliability  engineering. 

Actions  To  Be  Accomplished: 

•  Continuous  modification  of  failure  reporting  requirements  and  pro¬ 
cedures  to  obtain  information  essential  for  reliability  analysis. 
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SUBJECT 

UNREALISTIC 
DEVELOPMENT 
ENGINEERING  AND 
TEST  SCHEDULES 


FINDING  No.  12 

DEVELOPMENT  ENGINEERING  AND  TEST  SCHEDULES 
ESTABLISHED  EARLY  IN  A  PROGRAM  GENERALLY  PROVE 
TO  BE  OPTIMISTIC  AND  SELDOM  ARE  ACHIEVED. 


Symptoms  of  Basic  Deficiencies: 

•  Failure  to  meet  original  program  milestone  dates  by  a  wide  margin. 

-  New  schedules  show  ever-increasing  concurrent  achievement  of 
individual  milestones. 

•  Methods  of  recording  progress  are  changed  as  program  schedules 
are  slipped. 

•  Increased  program  costs. 

-  Contract  and  subcontract  cost  overruns. 

Fundamental  Causes: 

•  Planning  not  based  on  past  experience. 

•  Insufficient  analysis  for  determination  of  significant  factors  influenc¬ 
ing  engineering  and  test  schedules,  or, 

•  Insufficient  time  allotted  for  developing  schedules. 

•  Contractors  propose  unrealistic  schedules  in  order  to  be  competitive. 

•  Type  and  terms  of  contract  impose  no  penalty  on  contractor  for 
failure  to  meet  schedule. 

•  Added  requirements  imposed  by  Air  Force  without  changing  pro¬ 
gram  key  end  dates,  particularly  the  operational  dates. 

•  "Most  optimistic  possible”  dates  selected  as  technique  to  exact  best 
performance  of  personnel,  for  which  probability  of  accomplishment 
is  low  or  non-existent  at  outset. 

Actions  By  Contractors: 

•  Established  "unified”  test  program  concept,  reorganized  test  struc¬ 
ture  with  all  engineering  test  efforts  under  one  department,  includ¬ 
ing  planning  of  test  schedules. 

•  Applied  PERT  netting  to  development  engineering  and  test  sched¬ 
uling  to  better  identify  influencing  factors. 

•  Applied  past  experience  on  similar  programs  in  estimating  schedules 
initially. 

Actions  By  Air  Force: 

•  Implementation  of  a  competitive,  funded  program  definition  phase 
as  a  part  of  the  acquisition  cycle. 
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•  Source  selection  for  later  phases  of  acquisition  based  on  evaluation 
of  contractor  inputs  from  program  definition  phase. 

S. 

•  use  of  fixed  price  or  performance  incentive  type  contracts  for  de¬ 
velopment,  test  and  evaluation  phase  and  for  production  and 
deployment  phase. 

•  Require  implementation  of  the  PERT  system,  when  appropriate, 
for  new  programs. 

Actions  To  Be  Accomplished: 

•  AFSC  is  developing  a  System  Management  Manual,  which  includes 
program  definition/system  analysis  procedures.  Target  date  for 
publication  is  to  be  established  by  July  1963. 
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SUBJECT 


FINDING  No.  13 


INEFFICIENT 
UTILIZATION  OF 
ENGINEERING 
MANPOWER,  AND 
ASSIGNMENT  AND 
CONTROL  OF 
ENGINEERING  WORK 


THE  COMMON  FINDING  WAS  THAT ,  IN  GENERAL,  ENGI¬ 
NEERING  WORK  STATEMENTS  DID  NOT  SUFFICIENTLY  DE¬ 
FINE  THE  TASKS  AND,  THEREFORE,  WERE  NOT  ADEQUATE 
INSTRUMENTS  FOR  EVALUATION  OF  WORK  PERFORM¬ 
ANCE.  TWO  CIRCUMSTANCES  UNDER  WHICH  THIS  SITU  A- 
TION  OCCURS  ARE  OF  PARTICULAR  INTEREST  TO  THE 
AIR  FORCE;  NAMELY,  WHERE  THE  CONTRACTOR  HAS 
MULTI-PROGRAM  EFFORTS  WHICH  MAY  INCLUDE  A  MIX¬ 
TURE  OF  AF  AND  NON-AF  PROGRAMS,  AND  WHERE  THE 
CONTRACTOR'S  EFFORT  IS  LARGELY  A  SINGLE  AF  PRO¬ 
GRAM  BUT  A  SIGNIFICANTLY  LARGE  AMOUNT  OF  THE 
ENGINEERING  HOURS  IS  CHARGED  TO  A  VERY  FEW 
“CATCH-ALL”  ENGINEERING  WORK  ORDERS. 


Symptom*  of  Basic  Deficiencies: 

•  Inability  to  closely  relate  engineering  manhours  to  work  performed. 
-  Work  statement  not  sufficiently  definitive  in  many  instances;  how¬ 
ever,  some  may  be  completely  detailed. 

•  Inability  to  accurately  segregate  engineering  hours  expended  on  a 
given  program  from  total  departmental  effort. 


Fundamental  Causes: 

•  Decentralized  program  management  within  engineering  depart- 
ment(s). 

-  Functionally  oriented  rather  than  program-minded. 

-  More  concerned  with  having  flexibility  considered  necessary  for 
best  results  from  the  total  departmental  effort  on  all  programs. 

-  Less  concerned  with  precise  distinction  of  effort  expended  on  any 
one  program. 

•  Engineering  top  management  did  not  place  emphasis  on  definitive 
written  work  statements. 

-  Oral  description  of  details  was  considered  adequate  or  details  of 
work  required  was  considered  to  be  self-evident. 

-  Further  written  definitization  was  considered  superfluous. 

-  No  written  or  oral  instructions  were  issued,  however,  as  to  the 
most  essential  elements  of  work  or  which  categories  required 
greater  or  lesser  detail. 

•  Lower  level  supervisors  did  not  directly  use  statements  of  work  in 
measurement  of  group  work  output  nor  for  evaluating  technical  pro¬ 
ficiency  of  engineers  assigned  to  job,  although  manhour  charges 
were  made  to  the  work  order  numbers. 

•  Underlying  all  other  factors  in  utilization  of  engineering  manpower 
are  the  inherently  conflicting  demands  of  the  relatively  short-term 
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AF  programs  and  the  loag-term  company  interest  in  maintaining 
its  competitive,  technical  position  for  future  business. 

-  Rapid  manpower  build-up  initially  and  tail-off  at  program  eotn- 
pletioa;  proper  phasing  for  program  economy  in  AF  interest. 

-  Stable  engineering  force  in  best  interest  of  company. 

-  Potential  compromise  of  AF  program  interest  when  company 
efforts  include  both  AF  and  non-AF  programs  and/or  mixture 
of  fixed  price  and  cost  plus  fixed  fee  types  of  contracts,  all  in 
coo  junction  with  non-definitive  statements  of  work. 

-  Company  position  that  written  definitization  of  all  mh  not 
warranted  or  economical  in  every  case  and  most  efficient  day-to-day 
utilization  of  engineering  manpower  is  in  best  interest  of  all 
programs  has  some  validity;  however,  the  position  is  weakened  by 
lack  of  supporting  evidence  that  can  be  evaluated  by  AF. 

•  Long-term  interests  of  AF  and  industry  on  utilization  of  engineer¬ 
ing  manpower  are  more  in  agreement. 

-  In  AF  interest  that  a  number  of  companies  are  equally  com¬ 
petitive  on  future  programs. 

Actions  By  Contractors: 

•  Increased  definitization  of  engineering  tasks  by  enforcing  compli¬ 
ance  with  existing  policies  and  procedures. 

-  Most  companies  already  have  in  being  good  written  procedures 
for  assignment  and  control  of  engineering  tasks. 

•  Training  programs  have  been  started  to  provide  a  greater  capability 
at  all  levels  in  the  preparation  of  engineering  work  statements. 

•  Centralization  of  program  management. 

Actions  By  Air  Force: 

•  Required  centralized  program  management  for  large  AF  programs 
or  where  contractor  has  diversified  engineering  effort  involving  fixed 
price  and  CPFF  contracts  or  mixture  of  AF  and  non-AF  programs. 

•  Tighter  control  of  engineering  studies  and  analyses,  by  specific 
identification  of  effort  and  approval  by  Contract  Change  Notices; 
increased  use  of  design  problem  log  books  in  SPOs  indicating  nature 
of  problem  area,  task  number,  objectives,  estimated  engineering  man¬ 
hours,  actual  manhours,  schedule,  and  end  product,  i.e.,  ECP,  design 
change  prototype,  R&D  test  schedule,  qualification  test,  etc. 

•  Increased  surveillance  of  engineering  tasks  by  AFPRO/CMO  on 
periodic  basis. 

•  Required  better  coo  tract  definition  by  SPO  of  work  to  be  performed. 

Additional  Actions  Recommended: 

•  Joint  studies  by  the  Defense  Industry  and  the  AF  of  the  engineering 
manpower  problems  with  the  objectives  of  determining  whether  new 
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techniques,  policies  or  methods  can  better  reconcile  long-term  in¬ 
terests  with  short-term  program  requirements,  to  recommend  what 
policies  or  methods  are  needed,  and  to  recommend  implementation 
plans. 

•  As  a  part  of  the  negotiated  contracts,  the  AF  and  contractors  should 
work  out  a  "Chart  of  Accounts”  which  identifies  and  relates  major 
engineering  tasks  to  cost  and  schedule  as  a  means  of  uniform 
measurement  of  technical  progress. 
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SECTION  in 

Production  &  Quality  Assurance  Functions 

Finding  No.  Pago  No. 

14.  Inadequate  Facilities  Management .  51 

15.  Late  Deliveries  of  End  Items  .  53 

16.  Calibration  of  Test  or  Precision  Measuring  Equipment .  55 

17.  Quality  Assurance  Audits  not  Effective  .  57 
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SUBJECT 

INADEQUATE 

FACILITIES 

MANAGEMENT 


FINDING  No.  14 

LACK  OF  FACILITIES  MANAGEMENT  EMPHASIS  DELAYED 
DEVELOPMENT,  PRODUCTION  AND  TEST  PROGRAMS ; 
DUPLICATED  INVESTMENT ;  RESULTED  IN  LOW  RE¬ 
SOURCES  UTILIZATION ;  AND  INCREASED  COSTS  TO  THE 
GOVERNMENT  AND  INDUSTRY. 


Symptoms  of  Basic  Deficiencies: 

•  Low  facilities  utilization. 

•  Late  and  high  cost  production. 

•  Capital  equipment  charged  to  contracts  as  direct  costs. 

•  Retention  of  government-furnished  facilities  after  program  comple¬ 
tion. 

•  Failure  to  correctly  identify  and  report  idle  facilities. 

•  Poor  maintenance  of  equipment. 

■  Industrial  facilities  classified  as  special  tooling  or  test  equipment 
and  vice-versa. 


Fundamental  Causes: 

•  Facilities  screening  not  carried  out  aggressively  by  contractor  or 
Air  Force. 

•  Expansion  requirements  not  identified  in  time  to  provide  facilities 
for  production. 

•  Loose  inventory  control. 

•  Weak  enforcement  of  ASPR/AFPI  by  AF  and  contractor. 

•  Desire  of  contractors  to  maximize  government  facilities  investment, 
minimize  own  long-term  capitalization. 

•  Lack  of  top  management  concern. 

•  Misclassification  as  an  expediency  because  of  program  schedules 
and  funding  delays  anticipated. 

Actions  By  Contractors: 

•  Improved  screening  and  disposal  of  machine  tools. 

•  Improved  classification  procedures. 

•  Improved  organizational  distribution  of  tools  to  minimize  duplica¬ 
tion. 

•  Earlier  release  of  production/test  plans  with  facilities  impact. 

•  Increased  subcontracting  of  peak  loads. 

•  Top  management  review  of  government  as  well  as  company 
facilities  expansion  requirements. 


51 


AFSCP  375-2 


1  June  1963 


Actions  By  Air  Force: 

•  Project  "Tide”  established  by  Hq  USAF  to  re-examine  the  entire 
area  of  policies,  procedures,  and  industry  practices  of  acquiring, 
classification  and  control  of  specific  tooling. 

•  Intensified  review  of  facilities  acquisition,  utilization,  maintenance, 
storage,  and  disposition  by  Air  Force  Plant  Representative  Office/ 
contract  management  divisions  and  contract  management  regions. 

•  Establishing  central  control  of  Army,  Navy,  AF  idle  equipment 
inventory  under  Defense  Supply  Agency. 

•  Top  management  review  of  government  as  well  as  company  facili¬ 
ties  expansion  requirements. 

•  Facilities  costs,  in  support  of  production,  programmed  and  funded 
directly  from  within  the  total  system  program  budget.  Elimination 
of  all  direct  system  facilities  costs  from  current  separate  industrial 
facilities  budget  programs,  i.e.,  BP-1400,  BP-2400,  and  BP-8600. 

Actions  To  Be  Accomplished: 

•  AFP1  to  be  revised  as  result  of  findings  under  project  "Tide.” 
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SUBJECT 

LATE  DELIVERIES 
OF  END  ITEMS 


FINDING  No.  15 

LATE  DELIVERIES  CREATE  TRAINING  AND  LOGISTIC  SUP¬ 
PORT  PROBLEMS,  INCREASE  UNIT  COSTS,  AND  JEOPARD¬ 
IZE  OPERATIONAL  CAPABILITIES. 

Symptoms  of  Basic  Deficiencies: 

•  Unavailability  of  items  to  user. 

•  Undefinitized  letter  contracts. 

•  Increased  program  costs. 

•  Excessive  overtime. 

•  Shop  orders  behind  schedule. 

•  Continuous  expediting  of  items  through  shops. 

Fundamental  Causes: 

•  Poor  production  control. 

•  Basic  design  inadequate. 

•  Unrealistic  scheduling. 

•  Excessive  design  changes. 

•  Late  engineering  releases. 

•  Late  receipt  of  vendor  items. 

•  Delays  in  receiving  inspection. 

•  Material  shortages  to  production  line. 

•  Ineffective  in-house  expediting  procedures. 

Actions  By  Contractors: 

•  Revised  manufacturing  schedule. 

•  Requests  waiver  from  Air  Force  to  use  RAD  drawing  in  lieu  of 
Class  II. 

•  Aggressive  search  for  other  sources  for  components/materials. 

•  Established  automated  parts  control  system. 

•  Established  "line-of-balance”  and  other  advanced  production  control 
techniques. 

Actions  By  Air  Force: 

•  Amended  contracts,  establishing  realistic  delivery  schedules. 

•  Granted  waiver  for  use  of  R&D  drawings. 

•  Implemented  AFSCM  375-1  to  establish  configuration  management 
and  reduce  number  of  changes. 
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SUBJECT 


FINDING  No.  16 


CALIBRATION  OF  TEST 
OR  PRECISION 
MEASURING 
EQUIPMENT  DELAYED 


ALMOST  WITHOUT  EXCEPTION  THERE  WAS  LATE  CALI¬ 
BRATION  WITH  CONSEQUENT  IMPACT  ON  QUALITY  AS¬ 
SURANCE. 


Symptoms  of  Basic  Deficiencies: 

•  Overage  calibration  tags. 

•  Delivery  of  out-of-tolerance  hardware. 

•  Rejections  at  subsequent  production  stations  and  in  the  field. 

•  Excessive  rejects  of  out-of-tolerance  hardware. 

Fundamental  Causes: 

•  No  central  control  of  calibration  records  and  schedules. 

•  Inadequate  calibration  procedures. 

•  Insufficient  amount  of  calibration  equipment. 

•  Poor  discipline  and  enforcement  of  procedures. 

Actions  By  Contractor: 

•  Revised  procedures  and  emphasis  on  procedural  discipline. 

•  Coded  calendar  dates  for  recalibration  abandoned  in  favor  of  actual 
dates. 

•  Auditing  techniques  introduced. 

•  Central  control  and  records  established. 

Actions  By  Air  Force: 

•  Revoke  contractor  Quality  Assurance  system  approval  for  lack  of 
compliance. 

•  Increased  surveillance  by  AFQAR. 

•  Improve  contractual  requirements  by  use  of  MIL-C-45662A,  "Cali¬ 
bration  System  Requirements.” 
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SUBJECT 

QUALITY  ASSURANCE 
AUDITS  NOT 
EFFECTIVE 


FINDING  No.  17 

FAILURE  TO  AUDIT  QUALITY  ASSURANCE  PROCEDURES 
ALLOWED  SOME  QUALITY  ASSURANCE  SYSTEMS  TO  OPER¬ 
ATE  INEFFICIENTLY  AND  INEFFECTIVELY  WITHOUT  MAN¬ 
AGEMENTS  KNOWLEDGE. 

Symptom*  of  Basic  Deficiencies: 

•  Quality  procedures  not  followed. 

•  Production  frequently  failed  to  meet  contract  specifications. 

•  Excessive  rework. 

•  Rejectior  of  hardware. 

•  Repetitive  discrepancies/defects. 

•  Failure  reports  not  fully  used. 

•  Lack  of  communication  between  quality,  reliability,  engineering, 
manufacturing,  and  test  personnel. 

Fundamental  Causes: 

•  Inadequate  manning  of  QA  organization. 

•  Incomplete  support  of  top  management. 

•  Absence  of  written  quality  audit  procedures. 

•  Absence  of  trend  data. 

•  Quality  Assurance  and  audit  low  in  organization  structure. 

•  Quality  audit  results  not  enforced. 

•  Lack  of  follow-up  to  evaluate  effectiveness  of  corrective  action. 
Actions  By  Contractors: 

•  Improved  vendor  audit  system  to  concentrate  corrective  efforts  on 
poor  performing  vendors. 

•  Elevated  organization  placement  and  scope  of  responsibilities  of 
quality  audit  group. 

•  Increased  staffing  of  QA  departments. 

•  Written  quality  audit  directives  issued. 

•  QA  trend  charts  and  trend  levels  established. 

•  Quality  Review  Board  established. 
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Actiona  By  Air  Force: 

•  Continuous  surveillance  by  AFQAR  and  contract  management 
region  staff  quality  assurance  offices. 

•  Incorporation  of  "Quality  Audit”  requirements  in  contracts  by 
amendments  to  MIL-Q-9858,  "Quality  Assurance  System  Require¬ 
ments.” 

•  Provide  for  Air  Force  feedback  of  malfunction  data  (AFM  66-1)  to 
contractor. 
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SECTION  IV 

Purchasing  &  Materiel  Functions 

Finding  No.  Fage  No. 

18.  Inadequate  Competition  and  Price  Analysis  in  Award  of  Subcontracts. .  61 

19.  Late  Definitization  of  Letter  Contracts  with  Subcontractors  .  63 

20.  Inadequate  Subcontract  Management  .  63 

21.  Multiple  Purchase  Orders  for  Same  Item  .  67 

22.  Delayed  Termination  of  Subcontracts  .  69 

23.  Materiel  Control  Deficiencies  .  71 
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SUBJECT 


FINDING  No.  18 


INADEQUATE 
COMPETITION  AND 
PRICE  ANALYSIS  IN 
AWARD  OF 
SUBCONTRACTS 


THE  COMMON  FINDING  IS  THAT  INADEQUATE  COMPETI¬ 
TION  AND  PRICE  ANALYSIS  GENERATE  HIGH  COSTS,  ONE¬ 
SIDED  CONTRACT  TERMS,  AND  A  WEAK  POSITION  FOR 
FOLLOW-ON  PROCUREMENT. 


Symptoms  of  Basic  Deficiencies: 

•  Repetitive  purchases  from  seme  supplier. 

•  Many  single  or  sole  source  pu -chases. 

•  Frequent  expedited  procurements. 

•  Cursory  price  analysis. 

•  Delay  in  definituing  letter  subcontracts. 

•  Over-runs  on  CPFF  subcontracts. 

•  Excessive  subcontract  costs. 

Fundamental  Causes: 

•  Compressed  procurement  lead  times  caused  by: 

-  Unrealistic  delivery  schedules. 

-  Late  engineering  releases. 

-  Delays  in  processing  requirements  through  materiel  control. 

-  Lack  of  knowledge  concerning  lead  times  by  ordering  personnel. 

-  Concurrent  development/production. 

•  Failure  of  management  to  audit  or  enforce  policies,  procedures  and 
controls  governing  competitive  procurement. 

•  Insufficient  management  review  of  purchase  orders  and  price  analysis. 

•  Limited  experience  of  procurement  and  pricing  people. 

•  Cursory  purchase  order  evaluation  by  ACO  and  price  analysts. 

Actions  By  Contractors: 

•  Closer  purchasing/engineering  liaison  to  expedite  earlier  release  of 
longer  lead  time  items. 

•  Procurement  lead  times  emphasized  in  establishing  effcctivity  of 
proposed  changes. 

•  Purchasing  department  representative  participates  in  engineering 
change  activity. 

•  Electronic  data  processing  system  instituted  to  reduce  flow  time  of 
materiel  requirements  paper  work. 

•  Lead  time  report  published  for  major  items. 
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•  Top  management  project  of  developing  corporate  standards,  method¬ 
ology  and  techniques  for  cost  analysis  manuals. 

•  Vendor  Evaluation  Group  to  participate  in  source  selection. 

•  Cost  analysis  and  audit  procedures  improved  to  give  buyer  complete 
cost  history  records. 

•  Procurement  Review  Board  established  to  review  all  orders  over 
110,000  for  extent  of  competition,  cost  analysis,  and  justification  of 
single  or  sole  source  procurement. 

•  Expanded  training  program  introduced  in  purchasing  and  cost 
analysis  functional  areas. 

Actions  By  Air  Force: 

•  Continuous  evaluation  of  delivery  schedules. 

•  Increased  effort  on  the  part  of  AFPR  price  analysts  in  review  of 
subcontracts. 

•  Purchase  Systems  Manual  published  (AFSCM  70-3). 

•  Annual  purchasing  review  of  major  contractors’  purchasing  systems. 

•  Continuous  training  of  purchasing  system  analysts. 
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SUBJECT 

LATE  DEFINITIZATION 
OF  LETTER 
CONTRACTS  WITH 
SUBCONTRACTORS 


FINDING  No.  19 

EXCESSIVE  DELAYS  ARE  OCCURRING  IN  THE  DEFINITIZA¬ 
TION  OF  LETTER  CONTRACTS  WITH  SUBCONTRACTORS 
AND  VENDORS  REDUCING  ABILITY  OF  PRIMES  TO  CON¬ 
TROL  COSTS  AND  TO  NEGOTIATE  FAVORABLE  SUBCON¬ 
TRACT  TERMS.  FREQUENTLY  RESULTS  IN  NO  RISK 
FEATURES  AND  OTHER  DISADVANTAGES  OF  COST  REIM¬ 
BURSEMENT  AGREEMENTS,  WITH  FEE  AND  PROPRIETARY 
FEATURES  OF  FIXED  PRICE  CONTRACTS. 


Symptoms  of  Basic  Deficiencies: 

•  Contract  completed  shortly  after  definitization. 

•  Frequent  increases  in  dollar  limit  on  undefinitized  letter  contracts. 

•  Hasty  source  selection. 

Fundamental  Causes: 

•  Indefinite  or  ill -defined  statement  of  work. 

-  Insufficient  time  allotted  for  engineering  to  prepare  technical 
work  statements;  time  allotted  for  administrative  approval  cycle 
frequently  greater  by  factor  of  10  than  time  to  define  work  tasks. 

•  Poor  management  and  cost  controls  over  procurement  actions. 

•  No  periodic  accounting  of  expenditures  required  until  contract  was 
definitized. 

•  Failure  of  management  to  enforce  established  procurement  policies. 

•  Contractors’  opinion  that  delays  will  permit  price  reductions  by  the 
addition  of  quantities  to  existing  procurements. 

•  Lack  of  engineering — purchasing  coordination. 


Actions  By  Contractors: 

•  Restrict  the  use  of  letter  contracts  to  those  instances  where  procure¬ 
ment  lead-time  dictates. 

•  Established  a  limitation  of  40  per  cent  of  maximum  cost  data  ac¬ 
cumulation  at  which  point  the  contract  or  purchase  order  must  be 
negotiated. 

•  Education  and  training  program  for  defining  work  statement  pack¬ 
ages. 

•  Established  a  monthly  management  control  report  to  be  submitted  by 
each  Procurement  Department  for  letter  contracts  or  open  purchase 
orders. 

•  Established  training  course  for  engineers  in  preparation  of  technical 
work  statements. 
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•  Improved  exchange  of  information  between  engineering  and 
purchasing. 

Action*  By  Air  Force: 

•  Continuous  surveillance  by  purchase  method  analysts. 

•  Increased  emphasis  on  program  definition  prior  to  award  of  contract. 

•  Restricted  use  of  CPFF  contracts  and  subcontracts. 
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SUBJECT 

INADEQUATE 

SUBCONTRACT 

MANAGEMENT 


FINDING  No.  20 

SUBCONTRACTOR  MANAGEMENT  WAS  DEFICIENT  IN  THE 
AREAS  OF  PRICE  REDETERMINATIONS,  CONTROL  OVER 
EXPENDITURES,  AND  TECHNICAL  PROGRESS  EVALUATION, 
WITH  RESULTING  UNANTICIPATED  OVERRUNS,  AND  PER¬ 
FORMANCE/SCHEDULE  SLIPPAGES. 

Symptoms  of  Basic  Deficiencies: 

•  Delinquent  price  redeterminations. 

•  Subcontractor  cost  overruns. 

•  Little  use  of  learning  curves  in  negotiations. 

•  Unawareness  of  subcontractor  technical  and  production  problems. 

Fundamental  Causes: 

•  Late  submission  of  vendor  cost  data  and  untimely  audit  suport. 

•  Unrealiable  financial  reports. 

•  Failure  of  subcontractors  to  report  financial  data  required  by  the 
contractor. 

•  No  single  organizational  element  responsible  for  controlling  sub¬ 
contract  costs. 

•  Failure  to  follow  existing  procedures. 

•  Expedited  procurements. 

•  Inadequate  review  of  pricing  and  cost  estimating  of  subcontractor 
proposals. 

•  Inexperienced  buyers. 

•  Inadequate  technical  surveillance  of  subcontractors. 

•  Failure  to  examine  subcontractor’s  management  as  part  of  source 
selection. 


Actions  By  Contractors: 

•  Policies  revised  to  establish  the  requirement  for  determining .  the 
adequacy  of  the  subcontractor’s  accounting  system  prior  to  negotia¬ 
tion  of  subcontracts. 

•  Status  reports  revised  to  provide  more  timely  cost  control  informa¬ 
tion. 

•  Audit  group  reorganized  to  provide  adequate  audit  support. 

•  Subcontract  management  team  established  for  major  subcontracts 
with  engineer,  Quality  Control,  and  material  members. 

•  Formalized  source  selection  for  major  subcontractors. 
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Actions  By  Air  Force: 

•  Continuous  surveillance  by  purchase  method  analysts. 

•  Increased  technical/production  surveillance  of  subcontractors  by 
buying  activities. 

•  Delegation  of  secondary  contracting  responsibility  to  administrative 
contracting  officer  having  cognizance  over  subcontractor. 

•  AF-industrial  symposia  on  improving  subcontract  management. 
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SUBJECT 

MULTIPLE  PURCHASE 
ORDERS  FOR  SAME 
ITEM 


FINDING  No.  21 

SOME  CONTRACTORS  WERE  ISSUING  MULTIPLE  PURCHASE 
ORDERS  WITHIN  THE  SAME  TIME  PERIOD  FOR  LIKE 
ITEMS,  THUS  INCREASING  ADMINISTRATIVE  AND  UNIT 
COSTS. 

Symptoms  of  Basic  Deficiencies: 

•  Large  numbers  of  purchase  orders  per  dollar  volume  of  procure¬ 
ments. 

•  Variations  in  unit  costs. 

•  Excessive  documentation  requirements  for: 

-  Receiving  reports. 

-  payment  vouchers. 

-  Change  orders. 

Fundamental  Causes: 

•  Purchase  order  procedures  tied  to  need  to  segregate  costs  by  con¬ 
tract  and  to  automated  data  processing  system. 

•  Failure  to  consolidate  requirements. 

•  Failure  to  follow  established  procurement  policies  and  procedures. 

•  Lack  of  uniform  procedures  for  all  purchasing  departments. 

Actions  By  Contractors: 

•  Improved  communications  between  purchasing  and  materiel  control. 

•  Revised  scrap  processing  procedures. 

•  Proper  use  of  multiple  buy  cards. 

•  Study  extent  of  economies  through  combining  of  purchase  orders. 

•  Use  of  corporate  purchase  agreements,  blanket  orders,  open  end 
orders,  etc. 

Actions  By  Air  Force: 


•  Continuous  surveillance  and  assistance  by  purchase  method  analysts. 
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SUBJECT 

DELAYED 
TERMINATION  OF 
SUBCONTRACTS 


FINDING  No.  22 

LACK  OF  STRICT  PROCEDURAL  DISCIPLINE,  DELAYED 

NOTIFICATION  AND  SETTLEMENT  OF  TERMINATED  SUB¬ 
CONTRACTS,  AND  INCREASED  COSTS  TO  THE  AIR  FORCE. 

Symptoms  of  Basic  Deficiencies: 

•  Lack  of  control  and  aggressive  follow-up  by  the  buyer. 

•  Subcontractor  claims  not  settled  promptly. 

•  Increased  subcontract  termination  costs. 

•  Delays  by  buyers  in  updating  purchase  order  files  prior  to  forward¬ 
ing  to  termination  coordinator. 

•  Delayed  transmittal  of  termination  notice. 

•  Delays  in  subcontractor  notification. 

Fundamental  Causes: 

•  Lack  of  managerial  emphasis  on  the  termination  function. 

•  Contractor  decentralization  of  the  terminations  function  coupled 
with  lack  of  standardized  procedures. 

•  Slowness  by  the  Air  Force  in  (a)  requesting  audit  assistance,  (b) 
auditing  a  claim,  and  (c)  submitting  a  claim  to  the  Air  Force 
Review  Board. 

Actions  By  Contractors: 

•  Standardized  terminations  policies. 

•  Procedures  revised  to  provide  quicker  response  on  termination 
notices. 

•  Procedures  implemented  that  provide  instructions  to  subcontractors 
immediately  after  termination  notice. 

•  Regular  top  management  review  of  termination  status. 

•  EDP  system  to  reduce  flow  time  of  paper  work  in  materiel  control 
areas. 


Actions  By  Air  Force: 

•  Termination  functions  emphasized  and  placed  under  a  single  con¬ 
trol  point  in  contract  management  districts  to  assure  expeditious 
action  by  Air  Force  and  contractor  personnel  to  accomplish  timely 
settlements. 
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SUBJECT 

MATERIEL  CONTROL 
DEFICIENCIES 


FINDING  No.  23 

THIS  IS  A  COMBINATION  OF  SURVEY  FINDINGS  WHOSE 
IMPACT  IS  TO  DELAY  PRODUCTION  AND  DELIVERIES, 
STIMULATE  OVERTIME,  AND  INCREASE  SPACE  REQUIRE¬ 
MENTS  AND  COSTS. 

Symptoms  of  Basic  Deficiencies: 

•  Materiel  late  or  arrives  before  requirement. 

-  Inaccurate  stock  reports  and  inventory  records. 

-  Undependable  stock  levels. 

-  Materiel  requirements  established  without  knowledge  of  stock 
on  hand. 

•  Failure  to  coordinate  shipments  and  materiel  requirements. 

•  Materiel  control  delays  between  termination  notice  and  termination 
instructions  to  subcontractors. 

-  Inaccurate  lead  times. 

•  Value  analysis  program  in  contractors  Materiel  Department  in* 
effective. 

•  Supplies  not  standardised. 

•  Insufficient  consolidation  of  purchases. 

•  Government-furnished  equipment  lost,  or  location  and  condition 
unknown. 

•  Hurried  purchases. 

•  Delay  of  critical  items  in  receiving  inspection. 

Fundamental  Causes: 

•  Late  release  by  engineering. 

•  Inexact  procedures  and  assignment  of  responsibility. 

•  Insufficient  lead  time  on  purchase  requests. 

•  Unrealistic  end  item  delivery  schedules. 

•  Insufficient  control  over  materiel  transfers. 

•  No  top  management  audit  of  materiel  control  of  policies  and 
procedures. 

•  Insufficient  source  solicitation. 

•  No  evaluation  of  vendor  capabilities. 

•  Incomplete  purchase  order  data. 

Actions  By  Contractor: 

•  Feedback  system  of  statistical  data  to  focus  attention  on  substantial 
inventory  variations. 
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•  Materiel  Engineering  Coordination  Unit  within  the  Purchasing 
Group. 

•  Detailed  operating  procedures  for  materiel  control  formalized. 

•  Visitor  control  procedures  enforced  in  stockroom  and  warehouse 
areas. 

•  Full-time  stockroom  supervision  established. 

•  Audits  instituted. 

•  Employees  trained  in  data  processing. 

•  Established  GFP  record  and  control  center  with  responsibility  for 
property  accountability  and  reporting. 

•  Increased  effort  to  standardize  items  and  extend  program  to  other 
than  industrial  supplies. 

•  Publishing  a  periodic  lead  time  report  for  major  buy  items. 

•  Contract  negotiation  team  formed  for  all  major  procurements. 

•  Purchase  Order  Review  Board  established  to  measure  effectiveness 
of  purchasing. 

Actions  By  Air  Force: 

•  Follow-up  in  the  purchase  system  reviews  and  by  Air  Force  resident 
procurement  analysts. 

•  Ship  GFE  to  contractors  only  to  firm  requirements. 
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SECTION  V 

Product  Support  Functions 


Finding  No.  Pago  No. 

24.  Logistic  Management  Functions  not  Adequately  Recognized .  75 

25.  Technical  Data  Late,  Costly,  and  Inadequate .  77 

26.  Technical  Manuals  not  Validated/Quality  Substandard;  Deliveries  not 

Timely  for  Program .  79 
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SUBJECT 


FINDING  No.  24 


LOGISTIC 
MANAGEMENT 
FUNCTIONS  NOT 
ADEQUATELY 
RECOGNIZED 


A  MAJORITY  OF  CONTRACTORS  DID  NOT  HAVE  EFFECTIVE 
LOGISTICS  MANAGEMENT  ORGANIZATIONS.  AS  A  RESULT 
THERE  WAS  GREAT  VARIATION  IN  THE  TIMELINESS  AND 
QUALITY  OF  LOGISTICS  SUPPORT  PLANNING  DOCUMENT  A- 
TION  AND  DELIVERIES. 


Symptoms  of  Basic  Deficiencies: 

•  Late  delivery  of  and  inaccurate  technical  order  publications. 

•  Delinquent  delivery  of  spare  pans. 

•  Excess  Air  Force  material  for  support  of  contractor,  depot-level, 
maintenance  and/or  site  activation  tasks. 

•  Slippage  in  programmed  milestones. 

Fundamental  Causes: 

•  Company  precedent  followed  in  placement  of  functions  to  fit  exist¬ 
ing  organisation. 

•  Rapid  expansion  of  contractors’  programs. 

•  Lack  of  experience  and  appreciation  of  importance  of  logistic 
support. 

•  Lack  of  management  emphasis  of  support  functions. 


Actions  By  Contractor: 

•  Reorganized  to  assure  alignment  of  homogeneous  functions  under 
a  single  logistic  director. 

•  Strengthened  position  of  the  director  and  the  logistic  function. 

•  Implemented  procedures  to  clarify  interdepartmental  responsibili¬ 
ties  and  interfaces. 


Actions  By  Air  Force: 

•  Emphasis  to  contractors  on  need  for  improved  logistic  management. 

•  Management  responsibility  for  technical  order  validation,  and  AGE 
design,  requirements,  provisioning,  etc.,  assigned  to  SPO. 

•  Enforce  provisions  of  MIL-D-9412  and  MIL-M-26512;  MEP  71-650 
and  other  contractual  requirements. 
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SUBJECT 

TECHNICAL  DATA 
LATE,  COSTLY, 
AND  INACCURATE 


FINDING  No.  25 

THE  SURVEYS  INDICATE  A  GENERAL  LACK  OF  EMPHASIS 

BY  THE  AIR  FORCE  AND  CONTRACTORS  IN  THE  MANAGE¬ 
MENT  OF  TECHNICAL  DATA.  THE  RESULTS  ARE  FRE¬ 
QUENT  CHANGES,  HIGH  COSTS,  AND  DATA  INCOMPATIBLE 

WITH  DELIVERED  HARDWARE. 

Symptoms  of  Buie  Deficiencies: 

•  Inability  of  the  con  tractor  to  produce  technicel  publications  on  time. 

•  Inability  of  the  contractor  to  accurately  estimate  cost  of  development 
of  technical  data. 

•  Extremely  high  cost  for  technical  data. 

•  Constant  change  and  revision  of  technical  publications. 

•  Incompatibility  of  technical  data  (publications)  and  the  related 
hardware. 

•  Duplicate  contractual  requirements  for  data. 

Fundamental  Causes: 

•  Inability  of  the  Air  Force  (AFSC,  AFLC,  using  command,  ATC)  to 
define  requirements  for  data  at  an  early  stage. 

-  Lack  of  standardized  approach. 

-  Failure  to  agree  on  maintenance  between  interested  commands 
and  SPO. 

•  Lack  of  attention  and  emphasis  by  Air  Force  and  contractors  on 
importance  of  technical  data  compared  with  hardware. 

•  Contractor  split  management  responsibility  for  technical  data/tech- 
nical  manuals. 

•  Lack  of  coordination  between  engineering/design  and  tech  data 
people. 

•  Lack  of  methods  and  procedures  in  contractor  organizations  to  assure 
adherence  to  contract  provision  down  to  lower  levels. 

•  Lack  of  Air  Force  and  contractor  control  over  engineering  change 
proposals. 

•  Air  Force  frequent  changes  in  requirements  for  data  during  de¬ 
velopment. 

Actions  By  Contractor: 

•  New  operating  instructions,  emphasizing  responsibilities  in  techni¬ 
cal  data  areas. 

•  Organizational  changes  in  engineering  and  product  support  depart¬ 
ments  to  strengthen  capability  in  development  of  technical  data. 
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Action*  By  Air  Force: 

•  New  AF  System  Programs  will  include  a  funded  program  definition 
phase  which,  among  other  objectives,  will  be  a  forcing  function  for 
earlier,  systematic  identification  of  technical  data  requirements  and 
data  development. 

•  AFSC  established  as  the  single  command  responsible  for  establishing 
requirements,  funding,  procuring  and  reviewing  development  of 
technical  data  and  subsequently,  publications. 

•  Hq  AFSC  established  separate  functional  activity  with  responsibility 
for  management  of  technical  data. 

•  A  Dam  Management  Working  Group  established  in  July  1962  to 
examine  all  aspects  of  the  technical  data  problem,  including  proprie¬ 
tary  rights.  A  major  objective  of  the  group  is  the  development  of 
a  Basic  Data  Package,  consisting  of  minimum  data  requirements 
necessary  for  the  conception,  acquisition  and/or  operation  of  com¬ 
plete  system/ equipment.  (This  AFSC  group  has  close  liaison  with 
the  Aerospace  Industries  Association.)  In  the  development  of  the 
Basic  Data  Package,  the  group  is  progressing  along  these  lines: 

-  Identify  and  evaluate  AFSC  concepts  and  procedures  to  acquire 
control  and  use  the  data  required  for  system  programs. 

-  Select  a  standardized  approach  for  systematic,  time-phased  develop¬ 
ment  of  data  required  during  the  conceptual  and  acquisition 
phases. 

-  Develop  a  uniform  system  of  implementation. 

-  Identify  changes  required  in  existing  regulatory  documents,  in¬ 
cluding  regulations,  specifications,  military  contractual  provision¬ 
ing  documents,  technical  orders  and  contractual  exhibits. 

•  Air  Force  has  been  authorized  to  withhold  up  to  ten  per  cent  of 
contract  price  until  dam  is  delivered  or  deficiencies  are  corrected 
(when  contractor  is  negligent). 

•  AFSC  requires  a  joint  Air  Force/contractor  review  of  data  require¬ 
ments  on  at  least  one  system  program  at  each  division  (ASD,  BSD, 
ESD,  SSD)  each  year  to  insure  the  essentiality  of  existing  data  re¬ 
quirements  and  to  determine  that  the  procedures  used  are  effective. 

Actions  To  Be  Accomplished: 

•  Completion  of  follow-up  actions  resulting  from  Data  Management 
Working  Group’s  efforts — revision  of  requirements  document,  regu¬ 
latory  documents,  and  exhibits. 

•  Emphasize  and  manage  the  technical  data  programs  as  strenuously  as 
the  hardware  developments,  since  hardware  is  of  little  use  if  it 
cannot  be  operated  and  maintained. 

•  Establish  methods  of  accurately  costing  publications,  including 
technical  input  as  well  as  publication  and  printing. 

Additional  Actions  Recommended: 

•  Contractors  should  establish  company  policy  and  procedures  (organi¬ 
zations  if  necessary)  to  control  development  schedules  of  all  forms  of 
technical  data  and  insure  accurate  cost  accounting. 
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SUBJECT 

TECHNICAL 
MANUALS  NOT 
VALIDATED/QUALITY 
SUBSTANDARD; 
DELIVERY  OF 
TECHNICAL  MANUALS/ 
DATA  NOT  TIMELY 
FOR  PROGRAM 


FINDING  No.  26 

THE  SURVEY  FINDINGS  REVEALED  SUBSTANDARD  QUAL - 
ITY  OF  TECHNICAL  MANUALS  AND  DEFICIENT  OR  NON¬ 
EXISTENT  VALIDATION  PROCEDURES. 


Symptoms  of  Basic  Deficiencies: 

•  Long  delay  in  delivery  of  manuals  as  compared  to  hardware  (not 
concurrent  or  even  closely  concurrent). 

•  Inability  to  operate  and  maintain  equipment  with  instructions  con¬ 
tained  in  the  manuals. 

•  Excessive  changes  generated  during  verification  by  Air  Force. 

•  Excessive  AFTO  22  field  deficiencies  submittals. 

Fundamental  Causes: 

•  Lack  of  effective  coordination  between  contractors’  engineering  and 
technical  publications  organizations. 

-  Inadequate  inter-  and  intra-departmental  procedures. 

-  Indifference  on  part  of  engineering  in  providing  basic  technical 
data  and  support  to  the  technical  publications  people. 

•  Inadequate  configuration  management. 

-  Procedures  and  controls  were  concerned  with  the  hardware  only. 

•  Multiple/late  changes  in  requirements  by  the  Air  Force. 

•  Inadequate  "in-process”  reviews  and  guidance  by  the  responsible 
Air  Force  Program  Office,  or  lack  of  timely  planning  for  and  schedu¬ 
ling  of  technical  manual/technical  order  validations. 

•  Failure  of  the  contractor  to  adhere  to  contract  provisions  due  to 
lack  of  emphasir  and  adequate  instructions  within  the  company 
organization. 

Actions  By  Contractors: 

•  Revised  inter-  and  intra-departmental  procedures  for  development  of 
technical  data  and  the  planning,  identification,  scheduling,  publica¬ 
tion  and  validation  of  technical  orders  and  manuals. 

•  Some  contractors  have  unified  management  of  the  total  technical 
order  publication  function  where  this  did  not  exist. 

•  Construction  of  special  training/validation  facilities  at  plant. 
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Actions  By  Air  Force: 

•  AFSC  has  been  designated  the  responsible  Air  Force  agency  (or 
development  of  technical  orders/manuals  during  the  acquisition 
phase. 

•  AFR  80-14,  "Testing/Evaluation  of  Systems,  Subsystems  and  Equip¬ 
ments,”  is  in  process  of  revision.  This  regulation  contains  policy, 
procedures  and  general  test  objectives  which  include  the  determina¬ 
tion  of  necessary  data  for  handbooks/technical  orders/manuals  dur¬ 
ing  the  Category  I  test  phase  and  the  verification  and  validation  of 
all  necessary  technical  data  during  the  Category  II  test  phase. 

Actions  To  Be  Accomplished: 

•  Continue  implementation  of  AFSCM  375-1,  "Configuration  Man¬ 
agement  During  the  Acquisition  Phase,”  procedures  to  establish 
quick  reaction  to  and  greater  control  over  engineering  change  pro¬ 
posals,  and  to  provide  better  coordination  between  engineering/ 
design  changes  and  their  incorporation  in  technical  orders/manuals. 

•  Contractors  take  necessary  action  to  provide  better  coordination  be¬ 
tween  engineering/design  organizations  and  technical  publications 
groups. 

•  Greater  use  of  the  in-process  reviews. 

•  Exercise  of  greater  management  control  over  development/delivery 
of  technical  orders/manuals  by  both  Air  Force  and  contractor. 


Additional  Actions  Recommended: 

•  That  contractors  study  their  current  organization  and  procedures 
devised  for  management  of  technical  manual/technical  order  pro¬ 
grams.  There  is  seemingly  a  breakdown  in  translation  of  engineering 
and  design  data  into  useable  manuals  and  technical  orders,  which 
indicates  lack  of  coordination  between  engineering  design  people 
and  the  technical  publications  people  wh  »  actually  prepare  the  docu¬ 
ments.  On  an  overall  basis,  evidence  of  lack  of  validation  of  manuals 
and  hardware,  and  excessively  long  delays  between  scheduled 
delivery  and  actual  completion  of  the  manuals/technical  orders 
seems  to  point  to  the  need  for  more  strenuous  effort  to  control  man¬ 
agement  of  these  programs  equally  as  well  as  the  hardware  program’s. 
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CHAPTER  HI 
Summary  and  Conclusions 


Management  Deficiencies— 

A  WHAT  ARE  THEY? 

■  They  ere: 

•  All  occasions  of  omission  or  commission  (in  direction,  control,  conduct,  and  administration  of 
necessary  functions  and  processes)  which  compromise  attainment  of  programmed  coats,  schedules 
and  product  performance  objectives. 

A  WHY  DO  THEY  OCCUR? 

■  They  occur  because  of  imperfect: 

•  Planning — each  and  every  action  necessary  to  attain  program  objectives  was  either  not  recog¬ 
nized  or  not  properly  phased. 

•  Communications — policies  and  procedures  (instructions,  work-orders,  etc.)  do  not  define  authori¬ 
ties  and  responsibilities  sufficiently  to  direct,  control,  conduct,  administer,  or  integrate  the  char¬ 
acteristic  functions  and  processes. 

•  Discipline — Managerial  discipline  in  requiring  compliance  was  lax  in  areas  where  plans,  policies, 
and  procedures  are  adequate. 

•  Training  or  education — Wherein  greater  details  of  who,  why,  what,  when,  where,  and  how  of 
particular  actions  were  not  sufficiently  elaborated  for  people  responsible  at  lower  levels  for  con¬ 
trol,  conduct,  or  administration. 

•  Judgments — which  can  have  two  basic  effects:  (1)  an  impractical  program  where  costs,  schedules, 
or  product  performance  objectives  were  wrongly  determined  from  information  available  and 
cannot  be  practically  met  within  the  time  and  resources  allotted;  or,  (2)  where  wrong  decisions 
are  made  in  subsequent  plans,  communication,  discipline,  and  training  which  jeopardize  attain¬ 
ment  of  program  objectives  under  any  circumstances. 

A  WHAT  ARE  THEIR  CHARACTERISTICS? 

■  Generally,  they  are  encountered  in  defense  programs  of  large  magnitude  and  associated  with: 

•  Multi-agency  direction  and  participation. 

•  Incremental  funding. 

•  Contracting/subcontracting. 

•  Program  management  structure. 

A  System  design  and  integration  complexity. 

A  Cost  control  and  accounting. 

A  Purchasing  and  estimating. 

A  Technical  data. 

A  Configuration  management. 

A  System  hardware  assembly,  checkout,  and  turnover  to  Air  Force. 
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■  Occurrence  is  highly  sensitive  to  conditions  established  at  outset  of  program. 

*  Realism  of  program  objectives. 

*  How  program  is  contracted — prime/associate/subcontractor. 

*  SPO  and  contractor  organizations,  policies  and  procedures,  etc. 

■  Derive  from  the  inherent  nature  of  the  process(es)  to  which  they  pertain,  for  example: 

*  Detailed  design  poorly  integrated  as  system  because  of  the — 

—  Simultaneity  and  iterative  nature  of  the  design  process. 

—  Different  engineering  disciplines  and  technologies  that  are  involved. 

■  Rarely  isolated  to  a  single  cause  or  functional  area,  for  example: 

*  Late  definitization  of  letter  contracts. 

*  Technical  data  requirements,  development,  and  delivery. 

*  Cost  control. 

*  Reliability. 

*  Configuration  identification,  control  and  accounting  (see  chart  1  for  details). 

■  Frequently  they  are  caused  by  non-compliance  with  policy  and  procedures — both  contractor/Air 
Force,  and  a  laxity  in  managerial  discipline,  for  example: 

*  Make-or-buy. 

*  Calibration  of  precision  measuring  equipment. 

*  Classification,  acquisition,  and  disposition  of  facilities  and  special  tooling. 

*  Quality  control. 

*  Manpower  control. 

A  HOW  SERIOUS  ARE  THEIR  IMPACTS ? 

■  The  important  question  is  not  merely  whether  a  discrepancy  exists — but  how  serious  is  the  prob¬ 
lem.  The  degree  of  seriousness  is  a  function  of  the  specific  conditions  relevant  to  the  particular 
problem.  Depending  upon  the  relevant  conditions,  the  problem  may  be  a: 

*  MOUNTAIN — (Example) 

—  Technical  data  deficiencies  may — 

. .  Have  multi-agency  impacts. 

. .  Affect  AF  operation,  maintenance  and 
logistic  support. 

. .  Degrade  system  performance  and  reliability 
OR 

*  ICEBERG — (Example) 

—  Design  and  Engineering  integration  de¬ 
ficiencies  may  impact  upon: 

. .  Cost  control  and  program  schedules. 

. .  System  definition  and  end  item  identification. 

. .  Contract  and  subcontract  technical  workstatements. 

. .  Program  and  contract/subcontract  management. 

. .  Configuration  management. 

. .  Materiel  and  logistics  management. 

. .  Manufacturing  schedules  and  quality. 

. .  Identification  AGE  and  system  facilities. 

. .  System/equipment  interfaces. 
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. .  Technical  data. 

. .  System  assembly,  checkout,  and  integration. 

. .  System  turnover  to  AF. 

. .  System  performance/reliability. 

OR 

9  MOLEHILL-  (Example) 

—  Purchasing  systems  deficiencies:  (when  deficiencies,  though 
real,  represent  a  fraction  of  total  effort). 

. .  Inadequate  competition,  when  schedule  is 
over-riding. 

—  Excess  floor  space:  (when  facility  is  either 
company  or  government  owned  and  the 
curve  of  production  is  in  a  valley). 

A  IN  SUMMARY,  DEFICIENCIES  DO: 

9  Vary  in  degrees  of  seriousness. 

•  Vary  in  degrees  to  which  basic  causes  and  interrelated  effects  are  understood. 

•  Concern  some  management  areas  for  which  no  universally  accepted  criteria  for  measuring  per¬ 
formance  now  exists. 

•  Indicate  where  improvements  can  be  made. 

BUT 

A  THESE  DEFICIENCIES  DO  NOT  ALWAYS: 

9  Result  in  increased  costs. 

•  Reflect  a  degradation  in  program/contract  management. 

9  Justify  unlimited  application  of  resources  to  attain  perfection. 


A  WHY  DEFICIENCIES  ARE  CONSISTENTLY  POUND 

■  Perfect  judgment,  planning,  communications,  discipline,  and  training  are  unlikely  to  be  achieved 
simultaneously. 

■  Standards  of  performance  have  not  been  set.  (par  unknown). 

■  Contractor  and  AF  buying/contract  management  agencies  minimize  importance  of  problems. 

■  Insufficient  attention  by  contractor/ AF  management. 

■  When  no  penalties  assessed,  indifference  and  callousness  develop. 

■  Survey  teams  concentrate  on  same  items. 

■  Policies  and  procedures  do  not  integrate  semi-autonomous  activities  on  program  basis  (for  some 
applications). 

■  Compromise:  between  performance — schedules  vis-a-vis  following  established  procedures. 

■  Multi-agency  direction,  policy/procedures. 

■  Communication  barriers  between  Air  Force  and  contractors. 

■  Initial  program  decisions  and  conditions  affect  contractor’s  performance. 
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■  Top  management  emphasis  on  program  objectives  rather  than  detailed  procedural  compliance. 

■  Facts  of  life: 

*  People  sense  and  interpret  pressures  and  direction  with  varying  results. 

*  Differences  in  understanding  and  objectives. 

#  Discipline  varies  from  company  to  company,  from  supervisor  to  supervisor. 

•  Instability  of  work  force— changing  program  and  fluctuations  in  workload. 

•  Performance  criteria  established  by  DOD/AF  has  been  steadily  changing  over  the  past  few  years. 

•  Major  changes  require  period  of  evolution,  e.g.,  PERT,  configuration  management,  and  program 
definition. 


Air  Force/ Induttry  Management  May  Bring  About  General  Improvements  Through — 

A  INCREASED  ATTENTION  TO  MANAGEMENT  IN  ITS  TOTAL  PERSPECTIVE  BY: 

■  Appreciation  of  relationships  between  deficiencies — they  are  seldom  isolated  (see  chart  1  and  2). 

■  Increased  understanding  in  depth  of  the  interrelations  of  functional  processes. 

■  Recognition  that  initial  conditions  of  program  can  create  impossible  management  situations  later  on. 

■  Making  a  more  exacting  evaluation  of  the  management  structure,  policies,  and  procedures  to  elimi¬ 
nate  soft  spots  to  start  of  program — where  work  assignments  are  made  to  functional  organiza¬ 
tions,  insure  total  program  processes  are  integrated. 

■  Recognition  of  the  limitations  of  expedient  solutions  to  visible  management  problems. 

■  Avoiding  the  proliferation  of  sophisticated  topical  solutions  and  their  application  to  other  prob¬ 
lems  as  panaceas. 

■  Recognition  that  most  survey  findings  are  identified  with  functions  which  are,  or  should  be, 
under  surveillance  by  both  contractor  and  Air  Force  contract  management  people.  Therefore, 
improvements  lie  principally  with  the  contractor  and  local  Air  Force  management. 

A  NEW  POUCY  AND  PROCEDURES  BY  THE  AIR  FORCE 

■  Many  findings  reflected  conditions  that  were  due  to  inadequate  procedures  and  operation  under  the 
old  ARDC/AMC  arrangements,  such  as: 

V  Configuration  management,  where  split  management  existed,  which  allowed  duplication  of  pro¬ 
curement  and  change  actions.  New  regulations  and  manuals  have  been  issued  which  consolidate 
management  in  AFSC  for  acquisition. 

•  Technical  data  management,  which  also  had  split  management,  is  now  consolidated  under  AFSC 
for  acquisition. 

A  IMPROVED  TECHNIQUES 

■  Program  definition — where  a  funded,  competitive  phase  is  accomplished  for  the  purpose  of  better 
defining  the  program  by  means  of  cost  trade-off  studies  and  technical  analyses. 

■  System  definition  procedures/analysis  requirements  are  selectively  applied  throughout  the  acquisi¬ 
tion  cycle. 

■  PERT/Cost — which  will  aid  cost  estimating  and,  through  earlier  identification  of  cost  increases, 
will  provide  better  cost  control. 
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A  WIDER  DISSEMINATION  OF  SURVEY  FINDINGS/CORRECTIVE  ACTIONS 


■  Management  sympoaiuma — AF/Industry. 

■  Newsletters  and  other  media  are  being  used  to  provide  contractors  with  current  information  on 
repetitive  problems  as  well  as  new  techniques  and  policies  employed  by  the  Air  Force  and  industry. 

A  TRAINING  AND  EDUCATION  PROGRAMS 

■  Courses  are  now  being  provided  for  Air  Force  and  Industry  personnel  on  such  important  subjects 

as: 

•  Cost  estimating. 

A  Incentive  contracting. 

•  PERT. 

■  Some  contractors  have  initiated  courses  for  subcontract  management  and  workstatement  preparation 
for  buyers  and  engineers. 

■  Air  Force  has  established  a  system  management  course  for  training  senior  AF  officers  on  the  funda¬ 
mentals  of  program  management. 
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CHART  1 
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Toward  A  More  Useful  Pamphlet 

•  IMPROVEMENTS — What  arc  your  suggestion*  on: 

-  Format,  style,  or  presentation  of  problems? 

-  Impacts/effects  that  should  bit  included? 

-  Symptoms  and  causes  of  the  problems? 

-  What  was  done  to  correct  problems? 

We  welcome  your  comments  and  suggestions 

•  A  second  edition  may  be  published  depending  upon  the  general  response 
and  suggested  improvements. 

•  Address  your  comments  to: 

Assistant  for  Management  Surveys 
Hq  AFSC  (SCS-8) 

Andrews  AFB 
Wash  25,  DC 
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